Building MMview, a universal file viewer

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Post Reply
Message
Author
User avatar
Lobster
Official Crustacean
Posts: 15522
Joined: Wed 04 May 2005, 06:06
Location: Paradox Realm
Contact:

#91 Post by Lobster »

01micko wrote:
Lobster wrote:Am I at fault?
Yes.. RTFM .. "Remove fake gz extension" (stated in the 'download' table; "description" row).

I don't like the trend either but hey, I'm _not_ a trend setter.
:roll:

Read The Faking Manual? Should have guessed :oops:
Thanks for taking time off from the next Slacko64 development and guitar strumming in your 'man-shed' to answer . . .

Appreciate it :mrgreen:

Live Long and Puppy (traditional)
Puppy Raspup 8.2Final 8)
Puppy Links Page http://www.smokey01.com/bruceb/puppy.html :D

User avatar
Puppus Dogfellow
Posts: 1667
Joined: Tue 08 Jan 2013, 01:39
Location: nyc

hope you don't mind my efforts, Mochi

#92 Post by Puppus Dogfellow »

awesome again, Mochi!
MochiMoppel wrote:@stemsee MiView? Why not "MM MiEV"? Mitsubish Motors i-MiEV. Oh, I smell trouble.
MochiMoppel wrote:New question: Could someone suggest a better name? Quickview is already taken. .
i call it (and call it with) mmv--renamed it mmv so it's quicker from the terminal. (Double) eM Vee for MochiMoppel (file) Viewer is the rationale for The Bull With The Em Horns (yeah, they're more positioned like ears) Icon--pet gives both a dmv (Department of Motor Vehicles, speaking of mitsubishi (--and i was pretty fond of my 91 dsm gsx) and speaking of many, many files...) and an mmv version of the icon. that ring through the bull's nose is supposed to be in the shape of a directory folder/that directory folder thing's supposed to be a ring in da bull MV's nose.

the two entries end up in the file system menu as MochiMoppel File Viewer.
(i think it deserves two slots).

:cry: :D
Attachments
dabullmv-mmv-dmv.jpg
(2.8 KiB) Downloaded 805 times

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#93 Post by MochiMoppel »

Update 2017-04-06 (see also initial post)

F1 becomes a multi-purpose shortcut
So far F1 allowed to pull usage messages from executables. Additionally it now displays on-demand info for directories and archives. Sort of context sensitive shortcut.

Support for archives
This is experimental and I don't even know if this can be useful. F1 will list the files contained in .pet or other archives. It is supposed to work for .tar, .pet , .txz, .gz , .gzip, .bzip2, .deb. archives. It does not work for initrd.gz. I would ge grateful for any help to improve this feature since I don't know much about archives.

Directory info
F1 will add information about the disk usage of a directory's subfolders. Since subfolders are sorted by size this will make is easy to spot the biggest "disk eaters". IMHO a useful feature.

Run menu
F5 will run any selected executable in a console window. The title of the console window corresponds to the executed file name, which makes it easier to find the the right window when running multiple consoles. The windows can be closed with any key, not just the with the awkward key combos Ctrl+D or Alt+F4.
Attachments
mmview_archive.png
(147.88 KiB) Downloaded 792 times
mmview_directories.png
(116 KiB) Downloaded 774 times
mmview_runinconsole.png
(209.82 KiB) Downloaded 785 times

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#94 Post by greengeek »

Brilliant. Love the F1 functionality (and the prompts to try it...).
I notice if I click on something like /root/Startup/parcellite the view pane displays nothing. Would it be possible to display something like "symlink to.." or even just "symlink"?
Also with /root/Startup/network_tray there is no view pane display (possibly might be in future versions if a suitable viewer/decoder/interpreter is available) - but in the meantime might it be acceptable for the viewer to say something such as: "unsupported file type" or "C encoded file" etc etc?

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#95 Post by SFR »

MochiMoppel wrote:It does not work for initrd.gz. I would ge grateful for any help to improve this feature since I don't know much about archives.
Something like this should handle most of them:

Code: Select all

--- mm_view_20170406		2017-04-06 15:02:30.573360371 +0200
+++ mm_view_20170406_new	2017-04-06 15:19:08.486695165 +0200
@@ -205,7 +205,7 @@
 		;;
 
 	### ARCHIVES ###
-	*│*gzip*|*│*x-xz*|*│*x-bzip2*|*'.deb '*)		# .tar, .pet , .txz, .gz , .gzip, .bzip2, .deb
+	*│*gzip*|*│*x-xz*|*│*x-bzip2*|*'.deb '*|*'/initrd'*)		# .tar, .pet , .txz, .gz , .gzip, .bzip2, .deb, initrd*
 				MESSAGE=$'\n\tTry F1 to view package content'
 		;;
 
@@ -255,6 +255,19 @@
 		;;
 
 	### LIST ARCHIVE CONTENT ###
+	*'/initrd'*)
+		COMPRESSION=${vSTATUSBAR##*x-}						# '/initrd.gz │ application/x-gzip; charset=binary' => 'gzip; charset=binary'
+		COMPRESSION=${COMPRESSION##*application/}			# '/initrd.gz | application/gzip; charset=binary' => 'gzip; charset=binary'
+		COMPRESSION=${COMPRESSION%;*}						# 'gzip; charset=binary' => 'gzip'
+		case $COMPRESSION in
+			gzip)		CMND='gunzip -c -d'		;;
+			bzip2)		CMND='bunzip2 -c -d'	;;
+			xz|lzma)	CMND='unxz -c -d'		;;
+			lz4)		CMND='unlz4 -c -d'		;;
+			*)			CMND='cat'				;;			# not compressed
+		esac
+		${CMND} "$vCHOOSER" | cpio -ivt > $MSGFILE			# the output isn't really pretty with non-monospaced font
+		;;
 	*│*gzip*|*│*x-xz*|*│*x-bzip2*)							# extensions .tar, .pet , .txz, .gz , .gzip, .bzip2
 		COMPRESSION=${vSTATUSBAR##*x-}						# '/foo.pet │ application/x-gzip; charset=binary' => 'gzip; charset=binary'
 		COMPRESSION=${COMPRESSION%;*}						# 'gzip; charset=binary' => 'gzip'
There are also special cases out there, e.g. double compressed or ext2 images, but they're rather rare.
Theoretically, lzo(p) is also supported, but 'file -i' (at least in Fatdog) returns 'application/octet-stream' for it as well as for cpio, so it can't be used in the above scheme.

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
misko_2083
Posts: 114
Joined: Tue 08 Nov 2016, 13:42

#96 Post by misko_2083 »

greengeek wrote:Brilliant. Love the F1 functionality (and the prompts to try it...).
I notice if I click on something like /root/Startup/parcellite the view pane displays nothing. Would it be possible to display something like "symlink to.." or even just "symlink"?
Also with /root/Startup/network_tray there is no view pane display (possibly might be in future versions if a suitable viewer/decoder/interpreter is available) - but in the meantime might it be acceptable for the viewer to say something such as: "unsupported file type" or "C encoded file" etc etc?
Yes, like with the file command.

Code: Select all

file $FILE

Code: Select all

find $FILENAME -type f -exec file -b '{}' \; -printf '%s\n'
Like 3 years ago I made this little bash/zenity combo to analyse the content of usb drives. Some bits could be usefull (attach)[/code]
Attachments
ausb.sh.bz2
(1.76 KiB) Downloaded 207 times

bpuppy

Re. The multiviewer

#97 Post by bpuppy »

Just awsome!

I was longing for that same fonctionnality I used to have (ions ago) when I was using Total Commander's Quickviewer on an XP.

It's also quite incredible that we get it from a 20K script too!
It runs smoothly on PuppyTahr 605. I will also try it out on Fatdog 710.

Thank you so much MochiMoppel !!

slavvo67
Posts: 1610
Joined: Sat 13 Oct 2012, 02:07
Location: The other Mr. 305

#98 Post by slavvo67 »

Hi Mochi and others:

I think this is a great project but I noticed on a drive with a lot of files (lots of directories and subdirectories), it seems to slow down or hang to a point where it is not usable. I used one of the first scripts so I'm not sure if this has been addressed somewhere down the line.

As usual, this was in Quirky Xerus64.

Best,

Slavvo67

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#99 Post by MochiMoppel »

@greengeek When the target of a symlink is not displayed then not because it's a symlink but because the target is not viewable. And when nothing can be displayed I prefer to display nothing.

@SFR Thanks for the code. Seems that MIME application/x-gzip can be a lot of things. I don't like that :lol: I will use the code for my next update with a small change: I will use the busybox version of cpio. The output format is less ugly and can be easier turned into a palatable format for proportional fonts.

What I'm still not sure about are the MIME declarations in use. Having experienced problems with the 'x-' prefix, can I be sure that MIME is always labeled as 'x-gzip' and 'x-xz' and not 'gzip' and 'xz'? After all zip is reported as application/zip. I also don't know if the tar command really needs the -I option since it seems to sniff the compression program without it.

@misko_2083 The file command is already used for every selected file (see status bar).

@bpuppy Pleased to see that you like it and pleased to see that you are active again in the forum after more than 2 years. Yes, TC was great. I paid for it and still get updates but haven't used it for a long time. IIRC TC's Quickview had some weak spots. Search functionality was one of them. I'll try to make it better. Stay tuned.

@slavvo67 Already fixed.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#100 Post by greengeek »

MochiMoppel wrote:@greengeek When the target of a symlink is not displayed then not because it's a symlink but because the target is not viewable. And when nothing can be displayed I prefer to display nothing
Thanks for the reply. I see from your reply to misko_2083 that even for an unviewable file you display the "file" result in the status bar - would you also consider displaying something in the display pane which prompts the user to view the status bar? Maybe something such as "contents not readable" ?

Or is there perhaps some way to highlight the status bar text as an alternative so that it catches the users eye?
cheers!

User avatar
perdido
Posts: 1528
Joined: Mon 09 Dec 2013, 16:29
Location: ¿Altair IV , Just north of Eeyore Junction.?

#101 Post by perdido »

MochiMoppel wrote:Update 2017-04-06 (see also initial post)



Directory info
F1 will add information about the disk usage of a directory's subfolders. Since subfolders are sorted by size this will make is easy to spot the biggest "disk eaters". IMHO a useful feature.

.
Hi MochiMoppel,

This is an improvement.
One thing about it.
When I use the F1 to look at my /mnt/home/ directory I get no directory list in the right-side window due to my /mnt/home/ being a sym-link. Though /mnt/home/ does open and display contents in the left side window when clicked.

Also....
Are there plans to implement the [Esc] key to back out of things?

Thanks :D

.

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#102 Post by SFR »

MochiMoppel wrote:can I be sure that MIME is always labeled as 'x-gzip' and 'x-xz' and not 'gzip' and 'xz'?
At least in case of 'gzip', you can't - file (at least versions 5.04, 5.28 & latest) uses 'x-gzip', however Fatdog uses this patch from Debian:

Code: Select all

Description: Report gz as application/gzip
Author: Daniel Baumann <mail@daniel-baumann.ch>
Bug-Debian: https://bugs.debian.org/688886
Last-Update: 2015-09-18

--- a/magic/Magdir/compress
+++ b/magic/Magdir/compress
@@ -21,7 +21,7 @@
 #	* Produce shorter output - notably, only report compression methods
 #         other than 8 ("deflate", the only method defined in RFC 1952).
 0       string          \037\213        gzip compressed data
-!:mime	application/x-gzip
+!:mime	application/gzip
 !:strength * 2
 >2	byte		<8		\b, reserved method
 >2	byte		>8		\b, unknown method
So, probably all Debian-based distros (including some Puppy derivatives), have it, too.

But there can be more:
https://tools.ietf.org/html/rfc6713 wrote:Some applications have informally used media types including application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, and gzip/document to describe data compressed with gzip.
Perhaps using *gzip*) (or even *g*zip*) instead of gzip) in case statement would be more reliable...

Anyway, the bottom line is that application/gzip is the right one.
I also don't know if the tar command really needs the -I option since it seems to sniff the compression program without it.
Yeah, it shouldn't have problems with auto-detecting the used compression method.
Seems that MIME application/x-gzip can be a lot of things. I don't like that :lol:
Then you're gonna hate this - just checked in UExtract and files with the following extensions can be seen as application/gzip:
  • .adz
  • .apk
  • .bgz
  • .blend
  • .cgz
  • .cld
  • .cpgz
  • .cpio.gz
  • .cpio.gzip
  • .cvd
  • .cxarchive
  • .depot.gz
  • .dia
  • .dsl
  • .dz
  • .emerald
  • .emz
  • .epk
  • .fat
  • .gnumeric
  • .graphmlz
  • .gtp
  • .gz
  • .gzi
  • .gzip
  • .iar
  • .igz
  • .ipk
  • .jgz
  • .mpz
  • .nif
  • .oar
  • .obt
  • .ogz
  • .pet
  • .psz
  • .qpk
  • .qpr
  • .rub
  • .sdz
  • .sifz
  • .svgz
  • .tar.gz
  • .tar.gzip
  • .tg
  • .tgz
  • .unitypackage
  • .vbox-extpack
  • .vgz
  • .wrz
  • .x3d
  • .xcfgz
  • .xjtgz
  • .zabw
Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#103 Post by technosaurus »

all posix shells do globbing, so you can use application/*gzip in your case statement. This reminds me of some code I threw together to implement Xarchive in shell and gtkdialog. I only wrote the reading and formatting code for it, but it seems like that might be enough for your usage. I'm on my droid right now, so the link will have to wait, but it is linked to in goingnuts gtkdialog1 (backport to gtk1) thread.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#104 Post by MochiMoppel »

@greengeek As I already tried to explain: In principle there is no relation between symlink and viewability, hence no need to highlight, blink or sound anything when a symlink is not viewable. The only case where such relation exists would be a dead link, but this would be evident from the status bar and I don't intend to take additional measures for such case.

@perdido I'm not sure what you are after. The behavior you describe is normal and intended, but maybe not well understood and the "no subdirectories" message is irritating ( a not dereferenced symlink can't have subdirectories by definition). After lot of hair pulling I concluded that it would be best to avoid any ambiguities by treating links like empty directories: Just a hint that it's a symlink to <target> and no additional F1 info. And while I'm at it I will do the same for the /proc directory. Just a hint that it's a virtual file system. Avoids needless processing and delays for calculating useless information.

Regarding [Esc]: Plans? It's already implemented. The [Esc] key serves as the panic key for stopping autoplay and was introduced with Update 2017-02-04 ("To stop the audio: Press ESC key or change to non-audio file or close the window or dbl-click the file, which will stop ffplay and start to play the file in the default player.")

@SFR,technosaurus I think I'll have to tackle any problems resulting from this mess pragmatically. Knowing the specs of files inside an archive is nice but not terribly important. If MMview fails for any less common archive construction, it fails. Not the end of the world. Here is what I will do:
- throw an archive to tar and see if it can display the contents
- if not, feed it to gzip. Ideally the content is a single text file. MMview would display the content of the file, and that's the ultimate purpose of the viewer.
- if it wasn't a gzip file? Bad luck.

As a bonus I will add support for .iso images, .zip and .7z archives, and that should be it. If I screw up or something is really missing the users will let me know ... maybe.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#105 Post by greengeek »

MochiMoppel wrote:@greengeek As I already tried to explain: In principle there is no relation between symlink and viewability, .
Accepted. I think my post was badly written - i was not re-requesting symlink info to be displayed, just asking if file types (for non-viewable files) could be highlighted in some way.

User avatar
perdido
Posts: 1528
Joined: Mon 09 Dec 2013, 16:29
Location: ¿Altair IV , Just north of Eeyore Junction.?

#106 Post by perdido »

MochiMoppel wrote:
@perdido I'm not sure what you are after. The behavior you describe is normal and intended, but maybe not well understood and the "no subdirectories" message is irritating ( a not dereferenced symlink can't have subdirectories by definition). After lot of hair pulling I concluded that it would be best to avoid any ambiguities by treating links like empty directories: Just a hint that it's a symlink to <target> and no additional F1 info. And while I'm at it I will do the same for the /proc directory. Just a hint that it's a virtual file system. Avoids needless processing and delays for calculating useless information.
I was expecting too much, I understand now that the program will not display contents of a sym-linked directory in the right-side window.
Regarding [Esc]: Plans? It's already implemented. The [Esc] key serves as the panic key for stopping autoplay and was introduced with Update 2017-02-04 ("To stop the audio: Press ESC key or change to non-audio file or close the window or dbl-click the file, which will stop ffplay and start to play the file in the default player.")
.
Yes, I missed that. I was attempting to use [esc] to go backwards in the directory structure in the left-side window.

Thanks for taking the time to explain.

.

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#107 Post by MochiMoppel »

perdido wrote:I was attempting to use [esc] to go backwards in the directory structure in the left-side window.
Keyboard: Backspace
Mouse: Use button in "path-bar" (that's the button row above the file list, allowing a kind of a bread crumb navigation)

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#108 Post by technosaurus »

It looks pretty good so far, only a few pointers. Feel free to ignore since constructive criticism wasn't solicited.

If you want really fast MIME type detection by extension only first and then by magic values at an offset, you can use my simple MIMEtype program, but for the small number of filetypes a case statement is fine as long as you put the "multiple-dot" extensions first (Example follows).

Code: Select all

do_tgz_stuff(){
#template
}

do_gz_stuff(){
#template
}

case "$filename" in
  *.tar.gz|*.tar.gzip|*.tgz)do_tgz_stuff "$filename";;
  *.gz|*.gzip)do_gz_stuff "$filename";;
  #more here
esac
When case statements get really long it is helpful to put each in its own function. Humans tend to be able to take code in better if it fits on a single screen.


Here is another snippet that might be useful ... checks for the first match in a list of programs

Code: Select all

set_handler(){
	#$1 is the name of the variable you want to set
	#after shift the remaining $@ are executables you want to check for
	var="$1"
	shift
	while ! type -p "$1" 2>&1 >/dev/null; do
		shift #not found so go to next candidate
	done
	#this sets the variable to the value
	eval $var=\"\$1\"
}
#example usage
#note that there is no $ when you pass Do_sh to the function
[ "$Do_sh" ] || set_handler Do_sh loksh dash  hush ash sh bash
echo "$Do_sh is set as the preferred handler for .sh files"
Also rather than hardcoding urxvt, bash etc, you can use $TERM and $SHELL ... or for a text editor $EDITOR and web browser $BROWSER

BTW, I just finished compiling mupdf on fatdog64 and it has the ability to view or convert .xps, .pdf, .epub, .cbz and a few other formats and output them to png, pnm, pgm, ppm, pam, tga, pbm, pkm, pdf, svg or cbz ... it even comes with builtin javascript engine that allows you to do quite a bit of manipulation to build documents, but for this thread, the output to png and svg are the most significant (convert the first page of a pdf to an image for display). If you leave out the ridiculously large fonts (it's a PITA to do), the build is around 2MB.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#109 Post by MochiMoppel »

technosaurus wrote:If you want really fast MIME type detection by extension only first and then by magic values at an offset, you can use my simple MIMEtype program
Thanks. Currently I'm content with the speed. I'm more concerned with the correctness but haven't seen serious problems so far.
Also rather than hardcoding urxvt, bash etc, you can use $TERM and $SHELL ... or for a text editor $EDITOR and web browser $BROWSER
Yes, I can, but I shouldn't :lol: . Using these global variables (if that's what you mean and not locally assigned variables) I would end up in Slacko and maybe other Puppies with mp as $EDITOR and the (non existent) rxvt-unicode as $TERM. Even the global "Puppy variable" $DEFAULTTEXTEDITOR (in Slacko assigned to Geany) would be problematic since I can't be sure that the user hasn't changed his default editor after starting X. Hardcoding gives me control and avoids using programs the script is not designed for.
for this thread, the output to png and svg are the most significant (convert the first page of a pdf to an image for display)
IIRC converting pdf to an image can be done relatively easy with command line tools already available in Puppy, but then the question comes up if this is really worth it. However where I see a real potential are formats for which no suitable program is installed. In such cases an image could be a helpful work around and would be better than viewing nothing at all.

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#110 Post by MochiMoppel »

Update 2017-04-16 (see also initial post)

Some new functions and support for additional archive formats.

Line numbers
"Show line numbers" (shortcut F2) adds line numbers to text in the viewer pane. This feature in itself can be useful when viewing scripts etc., but beyond that it complements the new "Find again" feature (see below).

Refresh
For removing above line numbers the new "Refresh" feature (Shortcut F5) comes handy. So far the user had to select a different file, then reselect the current file to achieve a refresh/reset. F5 is common in browsers to reload a page, so it looks like a natural choice. The shortcut for "Execute selected file in console" had to move from F5 to Ctrl+F5 (which also better prevents accidental file execution).

Find
Frankly I never thought it would be possible to implement decent search capabilities in gtkdialog. Too many tools are missing, particularly a tool to highlight matching search results...no, Pango markup is not an option.

Finally I came up with a "Find" dialog (shortcut Ctrl+F), which allows 2 alternative display options, each of them with pros and cons, but both already much better than Total Commander's QuickView.

Extract matching lines
This is the default and very similar to Geany's brilliant "Find document usage" (In Geany select a word, press Ctrl+Shift+D and Geany will present a list with all lines containing this word). MMview diplays a similar list. It's not possible to jump to a specific line, but with F2 (show line numbers) it's possible to display all lines. F3 (Find again) returns to the extracted match lines. Particularly for large documents the F2 / F3 combo allows a fast review of search matches.

Use inline markers
All document lines remain displayed and matching search strings are marked with a fat unicode character. Not as good as the impossible highlighting, but not bad either. Here again the F2 / F3 combo comes handy. Hawk-eyed users will notice a ':' after numbers of matching lines and a '-' after numbers of not matching lines. That's the way grep shows results.

Find again
In most applications F3 is a shortcut for "Find next" and would jump to the next match within a document. Gtkdialog can't jump. "Find again" performs the search definded in the Find dialog, but not only for the current file. After selecting a different file F3 will perform the same search on the newly selected file. Again and again...until the user changes the pattern in the Find dialog.

A summary of the search is displayed in the status bar.

New archive formats
MMview now diplays contents of initrd.gz , .iso, .zip and .7z archives
Attachments
MMview-Find.jpg
(105.51 KiB) Downloaded 697 times

Post Reply