Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 22 Oct 2014, 19:56
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Unsorted
upx, the compressor for executables
Moderators: deshlab, Flash, GuestToo, Ian, JohnMurga, Lobster
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [12 Posts]  
Author Message
MU


Joined: 24 Aug 2005
Posts: 13642
Location: Karlsruhe, Germany

PostPosted: Wed 24 Aug 2005, 18:40    Post subject:  upx, the compressor for executables  

Hi,

somewhere in this board I read, someone uses Opera 5 statically compiled with QT.

The result should be a huge executable.

To minimize size, you might try upx, the compressor for executables: http://upx.sf.net

I use it to compress the wxbasic-interpreter.

example:
Consoleversion uncompressed: 150 kb, compressed: 50 kb
Gtk-version uncompressed: 5.8 MB, compressed: 1.8 MB

Mark
Back to top
View user's profile Send private message Visit poster's website 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15117
Location: Paradox Realm

PostPosted: Wed 24 Aug 2005, 22:23    Post subject:  

Very good advice - I have found upx very useful - the programs run faster when compressed, being smaller on the load into memory (but this I have never noticed so it is a small increase)

As a side question, Puppy uses a compression system (I am sure this has been thought out) would someone like to explain this?

_________________
Puppy WIKI

Last edited by Lobster on Wed 24 Aug 2005, 22:46; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
Bruce B


Joined: 18 May 2005
Posts: 11130
Location: The Peoples Republic of California

PostPosted: Wed 24 Aug 2005, 22:43    Post subject:  

According to my understanding a UPX compressed file uncompresses itself prior to loading in memory. It uses just as much memory as the uncompressed version.

The main advantage I know of off hand to compressing an executable binary file with UPX is it occupies less disk space. Or RAM space if that's where you are storing it.
Back to top
View user's profile Send private message 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15117
Location: Paradox Realm

PostPosted: Wed 24 Aug 2005, 22:49    Post subject:  

?

Are you sure - I have only used the Windows version - this actually compresses the executibles. Question

_________________
Puppy WIKI
Back to top
View user's profile Send private message Visit poster's website 
Bruce B


Joined: 18 May 2005
Posts: 11130
Location: The Peoples Republic of California

PostPosted: Wed 24 Aug 2005, 22:52    Post subject:  

> Are you sure - I have only used the Windows version - this actually compresses the executibles.

It compresses them quite effectively. It has to be decompressed to run. The decompression is seamless and you don't see it. Am I sure?

98% sure.
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11119
Location: Arizona USA

PostPosted: Wed 24 Aug 2005, 23:20    Post subject:  

If I understand correctly, here's how works in Knoppix: the info on the CD is compressed (personally, I think "condensed" is more accurate than "compressed," but I'll use compressed if it keeps everyone happy.) For the CPU to execute it, the info must be expanded (reconstituted) into the RAM, which is done (by the CPU, on the fly) as it is read from the CD. I've read that, in the case of Knoppix at least, the time saved by reading the info from the CD in compressed form more than makes up for the time it takes for the CPU to expand the compressed info. I have only the vaguest idea how a compression/expansion algorithm might work.
Back to top
View user's profile Send private message 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15117
Location: Paradox Realm

PostPosted: Wed 24 Aug 2005, 23:51    Post subject:  

Thanks Bruce

As usual I have not a clue wot I am on about - so thought it good to ask . . .
Does UPX put a header program in the executable that decompreses the (in effect a zipped type file)?

If that is an oversimplification - good - dumb it down further I say . . .

Smile

_________________
Puppy WIKI
Back to top
View user's profile Send private message Visit poster's website 
doopdoop

Joined: 28 Jul 2005
Posts: 48
Location: Magdeburg, Germany

PostPosted: Thu 25 Aug 2005, 00:58    Post subject:  

Puppy does basically the same things. The usr-filesystem from CD is a is a compressed read-only system (name squashfs, I think Knoppix uses the same). The algorithm is optimized for fast decompression with small overhead under these conditions. squashfs is implemented as an kernel module, the same way as reiserfs, ext2, iso9660 or msdos.
Compressing the executables on this fs does not reduce the real file size further( or not much, depending on the efficency of the algorithm), because they are already compressed. There are information thereotical limits for that.
But if you are really tight in the pup001 (which is an ext(2/3?), uncompressed), it can help.
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11130
Location: The Peoples Republic of California

PostPosted: Thu 25 Aug 2005, 02:32    Post subject:  

> in the case of Knoppix at least, the time saved by reading the info from the CD in compressed form more than makes up for the time it takes for the CPU to expand the compressed info. I have only the vaguest idea how a compression/expansion algorithm might work.

CD-ROM drives are not very fast compared to today's hard drives. Like way slow.

The decompression happens in RAM which is much faster than even a hard drive.

I'm not familiar with Knoppix much but the statement appears sound to me.

-----------------------

Back to Puppy, suppose some one wants to compress the binary exectables on the hard disk. I bet you would never notice the delay in the time it takes to decompress.

In theory the read heads could pick it up quicker because it is smaller. The decompression is very fast.

I've not noticed one ounce of difference in running a compressed exectable as compared to an uncompressed executable.

Also UPX can decompress the file you compressed to its orginal state. From this perspective one has nothing to lose by experimenting. The changes one makes don't have to be permanent.
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11119
Location: Arizona USA

PostPosted: Thu 25 Aug 2005, 11:06    Post subject: Squashfs  

doopdoop wrote:
Puppy does basically the same things. The usr-filesystem from CD is a is a compressed read-only system (name squashfs, I think Knoppix uses the same). The algorithm is optimized for fast decompression with small overhead under these conditions. squashfs is implemented as an kernel module, the same way as reiserfs, ext2, iso9660 or msdos.
Compressing the executables on this fs does not reduce the real file size further( or not much, depending on the efficency of the algorithm), because they are already compressed. There are information thereotical limits for that.
But if you are really tight in the pup001 (which is an ext(2/3?), uncompressed), it can help.
Thanks! I wondered what squashfs is, but figured it would take a long hunt through many websites before the light began to dawn. Your explanation, that it is in essence a compressed filesystem implemented as a kernel module, is very helpful to understanding what it is used for and how it is used.

Is that about right?

The initial Linux kernel which loads into RAM from the CD to begin the boot process obviously cannot be compressed on the CD, because the computer does not contain any program able to expand it. (Though, in theory, the BIOS could contain something similar to squashfs.) That initial kernel is what contains the squasfs module, which expands the subsequent compressed info that is read from the CD as the boot process continues.
Back to top
View user's profile Send private message 
doopdoop

Joined: 28 Jul 2005
Posts: 48
Location: Magdeburg, Germany

PostPosted: Thu 25 Aug 2005, 11:37    Post subject:  

Well, yes, partially right. The Bootloader (grunb, lilo, whatever) starts a part of the kernel, which can expand an initial ram disk (initrd.gz). This ram disk contains the modules (like squashfs, ext3 or SCSI, whatever). So, in fact the modules are compressed too, but with a different system.
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11130
Location: The Peoples Republic of California

PostPosted: Thu 25 Aug 2005, 13:58    Post subject:  

I'd say the rule for an option 1 install is do not compress anything in /usr

It will backfire on you. /usr is a merged file system with /root/.usr

Gnumeric for example is in /usr and not in /root/.usr by default. If you compress gnumeric, it (the compressed version) will show up in /root/.usr as a new file that was not there before. Suppose the compressed size is 1.5 mb, you have justed added 1.5 mb to pup001.

If you want to save space in /usr which is where a lot of executable files exist, make sure that the file exists in /root/.usr

If it already exists in /root/.usr then you will save space in pup001 by compressing it. Otherwise you will add space.

As far as my-roxapps and my-applications/bin are concerned you can save space by compressing them. This post applies only to /usr and the merged file system /root/.usr.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [12 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Unsorted
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0731s ][ Queries: 12 (0.0050s) ][ GZIP on ]