UExtract-4.2

Miscellaneous tools
Message
Author
User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#46 Post by SFR »

Hey Amigo

As for ffmpg/pdf, I understand your point and you almost convinced me, but I'd like to draw attention on an elusive difference: it's not verbatim "unarchive" utility, but more generic "extraction" utility.
TBH, if I called it, e.g. "Unarchiver" or similar, I'd have never thought to put such stuff into it.
Yeah, at the beginning I had only archives in mind, but over the time my guideline evolved to: "If the manner of processing this [format to be added] fits to (or at least doesn't abuse glaringly) the meaning of 'extraction' - put it in!".
And although this kind of logic has enslaved me, I'm gonna rethink this once more...and perhaps I'll follow what my heart says, not my mind. :lol:

Ok, I'm waiting for the diff, thanks. 8)
______________
mikeb wrote:
Very readable -nicely indented, etc.
Someone should buy Barry a 'tab' key for christmas :D

mike
:idea: :D :wink:

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#47 Post by amigo »

I think you could make two or three nice apps out of that code...
But, I've fallen into the same thinking -nearly every time I get good and started with a utility. There's always a tendency to over-reach with the scope of a project. Of course, you're gonna do just what you like with it anyway.

@mikeb You quoted what is, from me, high praise. This tool from SFR is maybe the most readable code I have ever seen on this site!

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#48 Post by mikeb »

Many a fun time I have had adding tabs to puppy scripts....

Hmm 'a get stuff out of anything tool'.... i catch your drift... I feared another pburn too but I am one for making simple gui's to do all that command line work simply because my memory would let me down.

Now if you could make it extract money...........

mike

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#49 Post by SFR »

I didn't plan to update it so quick, but since I found rather important bug, I won't be waiting, so here it is, v1.6:

- bugfix: if an archive contains one or more empty directories, they won't get unpacked
- added: warning at install time about missing dependencies (if any)
- removed .mht (it was never working too well, anyways)
- minor internal improvements

And a couple of new formats:
.cramfs (it was there before, in one of previous version, but got lost somehow)
.sqfs
.udf

Ok, now I'm really taking a break. 8)

First post updated!

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#50 Post by SFR »

SFR wrote:Ok, now I'm really taking a break. 8)
I lied, version 1.7:
- fixed typo (was 'unzx' instead of 'unxz')
- all 'xz -d' occurencies replaced with 'unxz' (for consistency)
- animated GIFs extraction (using gifsicle)
- can extract "naked" (extensionless, i.e. already gunzip'ed) initrd
- more 'fool-proof' mechanisms
- got rid of awk dependency
- some internal rearrangements

First post updated!

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#51 Post by SFR »

Update, version 1.8:

New extensions:
.flf (figlet font)
.ima
.msu
.oxt
.pak
.tarz
.themepack (Win7's)
.tz
.vbox-extpack

New formats:
- can extract Mozilla's .mar archives.
- can extract El Torito 1.2M/1.44M/2.88M boot image from within bootable ISO.
Will be saved as ./0_UExtract_BOOTIMAGE/FLOPPY.img (so won't get mixed with "normal" content) inside the extraction directory.
Of course such FLOPPY.img can be (in most cases) extracted further via UExtract.

The above pair is handled by my own routines, working on similar principles as exploderpm.

First post updated!

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#52 Post by amigo »

Are you keeping any notes about what the filetypes are?

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#53 Post by SFR »

Nope - most of them (origin) I remember.
But this site partially helps, when I can't recall: http://www.fileinfo.com/.

Do you have any particular filetype(s) in mind?

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#54 Post by SFR »

Update, version 1.9:

New formats/extensions:

.2fs|.3fs|.4fs (savefiles - both plain & encrypted)
.cpgz (same as cpio.gz)
.gz2 (misnamed bz2 file, lol)
.tar.gz2 (ditto)
.gzi (same as .gz)
.gzip (same as .gz)
.ipg (iPod game...not tested, but it's rumored that it's just a zip)
.jgz (gzipped Javascript file)
.pcv (MozBackup profile backup. Really - zip)
.piz (renamed .zip)
.psz (gzipped PostScript file)
.shr (same as .shar)
.taz (same as .tarz, .tz, .tar.Z, etc.)
.tlzma (was .tzma before, corrected)
.uue (same as .uu)
.xar (handled by 7z, so bye-bye permissions and stuff. :( )

.cfs (Compact File Set) - only basic & unencrypted!
http://www.pismotechnic.com/cfs/ wrote:Basic CFS files are compatible with ISO files, and can be read by many existing applications and systems.
.package (Autopackage format)
This one is handled manually (not via 'package_name.package --extract') due to lack of '--bytes' option in 'tail' in Slacko (and probably other Puppies).
Extracts both metadata and payload into a separate subdirs.
Doesn't verify MD5 (if something's wrong, appropriate unpacker will shout).
I obtained two of such packages and it works, but can't guarantee 100% reliability.

.pkg (Mac OS X Installer Package)
Two phases:
1. Extracts the main package (it's xar) using 7z
2. Extracts all 'Payload' and 'Scripts' subarchives using gunzip & cpio and removes them.

.txt - now UExtract can extract separate characters from a text file! [/jk] :wink:

What else:
- For consistency and better dependency checking 'tar j/z/J' changed to 'bunzip2/gunzip/unxz | tar'
- I noticed that encrypted zip archives created with 7z cannot be unpacked with 'unzip'.
So now, if extraction of 'zip' related archive fails, 7z (if present) will be used.
- fixed output of 'uextract -d | more'
- some fixes & cleanings

First post updated!

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#55 Post by Ted Dog »

could you look into porteus XZM modules, I copy/rename them as sfs and I can look into them but not merge them as sfs files, I think they may have a config.file run to merge them in.

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#56 Post by SFR »

Hey Ted

UExtract already supports extracting of .xzm modules (using unsquashfs4).

As for merging/creating them, I don't know too much about that process; did you check how it's done in /opt/porteus-scripts/dir2xzm script?

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#57 Post by Ted Dog »

Nope But thanks now I know where to look, :wink: a porteus spin has worked out the MESH networking, and I'm trying to get its unique xzm module to be useable by puppy.

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#58 Post by SFR »

Oh, now I see what you're trying to achieve. Hmm, when all the contents are already extracted, it would be just a matter to use dir2pet or dir2sfs, I guess.
But the question is, if the "guts" are compatible enough with Puppy. :wink:
Anyway, good luck!
__________________

Darn, I just discovered what a mess those .run installers really are - each one have different option for extracting!
And although I kinda improved the routine and now it supports amd-driver.run, nvidia-driver.run and VirtualBox.run installers, I decided to withdraw this ability for now - who knows how many more are out there and how would they behave..? :roll:

Version 1.10 uploaded.

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
Ted Dog
Posts: 3965
Joined: Wed 14 Sep 2005, 02:35
Location: Heart of Texas

#59 Post by Ted Dog »

I think its near enough... The are using pBurn as is and not even bothering to change its puppylinux name in the help! The may have even choicen a p... named linux so Pburn, Pmusic, Pmount etc sound native... :wink:

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#60 Post by SFR »

Update, version 1.11:

More...
.azw2 (Kindle Active Content App File - zip)
.cgz (same as .cpio.gz)
.cvd (ClamAV virus database. Really - tar.gz, starts @ 0x200)
.fpl/.vfd (floppy images)
.gadget (Windows sidebar gadget - zip, or sometimes cab)
.gem (RubyGems Package - tar & tar.gz)
.imz (zipped .ima)
.ipa (iPod application - zip)
.ipsw (iPod/iPhone Software Update File - zip. Contains encrypted .dmg files - 7z can't handle them, unfortunately).
.ita (IconTweaker Theme Archive - zip)
.obt (Openbox Theme - tar.gz)
.squashfs (same as .sqfs)
.studyarch (The Study Archive Format - zip)
.tazpkg (SliTaz package - cpio & cpio.lzma)
.tcz (TinyCore package - squashfs)
.thm (Sony Ericsson Theme File - tar)
.u3p (U3 Smart Application - zip)
.utz (UIQ Theme Package - yep, it's zip)
.widget (Microsoft Windows Mobile Widget - zip again)
.xap (Silverlight Application Package - what a surprise...zip)

.gifar
It's a malware, lol.
Extracts both .gif and .jar parts from within a file.
Well, actually such extension doesn't really exist - what haxor would name it like that anyway?
It'd be like "Hey everybody, this file contains malicious code."
But if anyone wants to create one, for testing purposes:

Code: Select all

cat some_picture.gif some_app.jar > pic_and_app.gifar
Such a "picture" still can be viewed and, to execute the .jar part, use 'java -jar pic_and_app.gifar'.
Quite a fun to have two functional things in one piece.

What else:
- got rid of pet2tgz dependency - now .pets are being unpacked directly
- minor fixes

First post updated!

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#61 Post by amigo »

Most .run files are created using makeself, so you should be able to filter those out. Otherwise, you've probably covered the majority of cases for the drivers...

Thanks for noting both the description and real type of file for the last batch of additions! Have you added rusty-bolt-extraction yet? I haven't found a really dependable way to extract them yet, but I know you can create them with tourque-wrench-wait-50-years...

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#62 Post by mikeb »

Have you added rusty-bolt-extraction yet? I haven't found a really dependable way to extract them yet, but I know you can create them with tourque-wrench-wait-50-years...
The hot spanner was a favourite but a pair of good quality mini mole grips could save the day with some fiddly problems. Of course a good long wd40 hot tub can relieve some situations...

mike

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#63 Post by SFR »

Don't have one atm, but cat rusty_bolt_and_nut | tr -d '[:rust:]' | tar xf - should do the job as well. :wink:
amigo wrote:Most .run files are created using makeself, so you should be able to filter those out. Otherwise, you've probably covered the majority of cases for the drivers...
...or maybe pull them out manually?
This one behaves promising and should be safe enough:

Code: Select all

#!/bin/bash

# Manually lists contents of .run installers (tested: amd/nvidia/vbox)

ARCHPATH="$(realpath "$1")"; [ -f "$ARCHPATH" ] || exit 1

for DUMMY in DUMMY; do
  OFFSET=$(grep -a -m1 '^skip=' "$ARCHPATH" | tr -cd '[:digit:]')
  [ $OFFSET ] && { METHOD="-n +$OFFSET"; break; }

  OFFSET=$(grep -a -m1 '^filesizes=' "$ARCHPATH" | tr -cd '[:digit:]')
  [ $OFFSET ] && { METHOD="-c $OFFSET"; break; }

  echo "Unsupported .run installer"; exit 1
done

TYPE=$(tail $METHOD "$ARCHPATH" | file -b -)

case $TYPE in
  *'POSIX tar archive'*)        tail $METHOD "$ARCHPATH" | tar tvf -            || exit 1 ;;
  *'xz compressed data'*)       tail $METHOD "$ARCHPATH" | unxz | tar tvf -     || exit 1 ;;
  *'gzip compressed data'*)     tail $METHOD "$ARCHPATH" | gunzip | tar tvf -   || exit 1 ;;
  *'bzip2 compressed data'*)    tail $METHOD "$ARCHPATH" | bunzip2 | tar tvf -  || exit 1 ;;
  *)                            echo "Compression method unknown, exiting...";     exit 1 ;;
esac
But I wish find and test more types of those .run monstrosities, before including it back into UExtract...

Oh, and I just now found out that 'GNU tar' automagically recognizes compression format, so it would be enough to use only 'tar xf' in most cases...but I think I'll keep stick with piping.

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#64 Post by amigo »

Yes, tar guesses the format from the name suffixes, but my earlier tip about using --use-compressor is better since it lets you second-guess tars abilities -like for pets which are really tgz archives.

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

#65 Post by SFR »

Update, version 2.0:

New extensions:
.dbk (Sony Ericsson Mobile Phone Backup File - not tested, but it's a zip)
.dotx (Word Open XML Document Template - zip)
.eftx (Office 2007 Theme Effect File - zip)
.emerald (Emerald theme - tar.gz)
.emz (Windows Compressed Enhanced Metafile - gz)
.gg (Google Desktop Gadget File - zip)
.ksf (KMPlayer Skin File - zip)
.mct (MSN Messenger, Packaged Wink File -cab)
.nex (Opera's Navigator Extension format - handled by 7z)
.oex (Opera Extension - zip)
.onepkg (Microsoft OneNote Package File - cab)
.potx (PowerPoint Open XML Presentation Template - zip)
.thmx (Office 2007 Theme File - zip)
.wlz (Winamp Language Pack - zip)
.wmz (Compressed Windows Media Player Skin - zip)
.xlam (Excel Open XML Macro-Enabled Add-In - zip)
.xltx (Excel Open XML Spreadsheet Template - zip)
.xsn (InfoPath Form Template File - cab)
.mp3/.wma - extracts album art, if any. So far works only with Slacko's ffmpeg, but not with e.g. Precise's avconv.

.run has been restored (the manual, safe routine)
Tested with:
- amd-driver-installer-catalyst-13.1-linux-x86.x86_64.run
- LinReaper0.87.run
- NVIDIA-Linux-x86-319.49.run
- tremulous-1.1.0-installer.x86.run
- VirtualBox-4.2.18-88780-Linux_x86.run
Won't work with 'binary' .run installers (e.g. mplabx-ide-v1.run)...grr, they should have named it as *.bin IMO.
Also, won't work with WMware-player installers.

.j - removed. My bad, it's a JAR Archiver's format, not related to JAVA's .jar.
I obtained an old JAR Archiver (works in DosBox) and created a test archive, but even 7z can't unpack it...

Other stuff:
- Added menu entry (under Utility) - if launched this way, Xdialog's fileselector will pop-up (only one file at a time can be chosen/extraced).
Actually, I've added it to simplify using UExtract as a right-click option in other file managers, like PCManFM or SpaceFM.
The user himself has to take care of associating appropriate filetypes with UExtract, e.g. in PCManFM:
right-click a file -> Properties -> Open with -> Customize -> Utility -> UExtract
BTW, if you don't want to have it as a default handler (launched by left-click on a file), expand "Open with" list again and choose another (if exists).

- Custom icon; since I can't draw at all, I assembled one using Vovchik's Pikona.

- If present, Gtkdialog's VTE feature will be used, instead of xterm. Slacko >= 5.4 (or >=5.5?) has it.

- /usr/local/apps/UExtract is now 100% self-containable ROX-App.
Drag'n'dropping multiple files onto it is allowed.
/usr/local/bin/uextract is now just a symlink to /usr/local/apps/UExtract/bin/uextract

- Added brief 'Filetypes Info' help page (dedicated to Amigo ;)) - can be viewed by right-clicking the UExtract's ROX-App dir (again, it's /usr/local/apps/UExtract, but it's better to create a shortcut on the pinboard).

First post updated!

Greetings!
Attachments
Screenshot.jpg
(102.63 KiB) Downloaded 1352 times
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

Post Reply