How to stuff 7% more material into your sfs's.

[rufwoof]

a bit longer to create, but are perhaps a bit snappier to
unpack to RAM when you load your Puppy

LZ4 compresses less tightly (larger size), but decompresses much quicker than both gzip and xz. It's a trade off between disk IO speed and processing (decompression) and ram transfer speed. LZ4 when using multi-cores can throw out data at ram transfer speeds (extremely fast).

Reading more disk IO/decompress quickly can be quicker than less disk IO (tighter compression)/slower output (decompression).

Also bear in mind that once a program has been read in once during a session, most of that will remain memory bound (cached), such that subsequent re-runs of the same program/libs will already be in memory (cache). Especially in the case of puppy that might be say 500MB uncompressed in total, running on a PC that perhaps has 2GB or ram. Rather than loading the entire pup into ram at startup (slower bootup), just loading files as and when required will load less (gparted for instance might never even be used during a session). Slower to initially load a program (such as gparted if invoked during a session) than having it already available in memory, but no difference for subsequent re-runs of the program.

Generally not loading puppy into ram at startup, using lz4 compressed sfs's is the better overall choice IMO. Maximising xz compression will tend to see a slower system by comparison whilst IO is more predominant, but thereafter the two tend to compare anyway (once cached).

As a example I run Debian frugally with a save partition (rather than a save file/folder) and comparing having all of that inside a sfs to having all of the filesystem in the save partition (non compressed) sees very little difference between the two. For me, having all of the data in the save partition leaving a empty main sfs means that I can also boot that as though it was a full install. There is a noticeable difference between booting full versus frugal however as with frugal all changes are being recorded in memory, not back on disk (I personally like frugal as you can just reboot without saving to undo all changes made during the session i.e. I like to keep the main system pristine/factory-fresh - I only ever tend to boot as though full when I apply updates, and then immediately reboot back to frugal again afterwards).

[greengeek]

LZ4 compresses less tightly (larger size), but decompresses much quicker than both gzip and xz.

Any chance you could suggest a good default syntax to build an sfs from a directory using LZ4? (and do you have any idea if LZ4 is understood by Slacko?)

[nic007]

Some more testing. Best xz compression achieved with: -comp xz -b 1048576 -Xbcj x86

Results compressing Racy base sfs:
gzip = 142MB
-comp xz (default setting) = 121MB
-comp xz -b 1048576 -Xdict-size 100% = 109MB
-comp xz -b 1048576 -Xbcj x86 = 105MB

Compression slowish, decompression time not affected. Good if you are only going to compress once or seldomly.

[musher0]

Some more testing. Best xz compression achieved with: -comp xz -b 1048576 -Xbcj x86

Results compressing Racy base sfs:
gzip = 142MB
-comp xz (default setting) = 121MB
-comp xz -b 1048576 -Xdict-size 100% = 109MB
-comp xz -b 1048576 -Xbcj x86 = 105MB

Compression slowish, decompression time not affected. Good if you are only going to compress once or seldomly.

Hi nic007.

Your memory came back, I see!!!

"Oh, sh?t." That's a 35% space savings over gz. Wow.

Bravo nic007.

[greengeek]

Some more testing. Best xz compression achieved with: -comp xz -b 1048576 -Xbcj x86

Thanks for the parameters. Using this gave me the following comparison for my base sfs:

454MB using gz method

351MB using Mushero xz syntax

335MB using nic007 xz syntax

Unfortunately both of the xz methods create an issue for me when the system tries to run my firewall setup script in /root/Startup so I assume I must have a delay in unpacking something at a critical moment.

Not a big issue and probably something I can find a way around (I think it is due to the fact I use xdotool to generate a keyboard action during the firewall script. Bad practice really).

The other issue for me is that the xz methods both take 30 seconds longer than gz to boot my machine but then i am working with quite a large sfs so maybe not a problem for others.

The significant space saving does make the xz syntaxes useful for making best use of storage space.

[rufwoof]

LZ4 compresses less tightly (larger size), but decompresses much quicker than both gzip and xz.

Any chance you could suggest a good default syntax to build an sfs from a directory using LZ4? (and do you have any idea if LZ4 is understood by Slacko?)

I suspect its not in Slacko.

It has to be built into the kernel as well as having updated squashfs-tools ... in which mksquashfs will typically have just -comp lz4 (basic choice) or -comp lz4 -Xhc (higher compression choice (decompression speed isn't any slower, but in decompressing less due to higher compression will throughput quicker)). Compression ratio isn't good. Higher speed comes with less complexity (less compression).

mksquashfs --help will show if the squashfs-tools version you have supports lz4, but the kernel might not necessarily also support it.

[greengeek]

mksquashfs --help will show if the squashfs-tools version you have supports lz4, but the kernel might not necessarily also support it.

Thanks - looks like only gzip and xz available. cheers!

[OvrMaHd]

I would like to thank all of the contributors of this thread. Has helped tremendously, to creating functioning Make SfS, from a plethora of obscure options.

Ovr

[drunkjedi]

Hey people,

Anyone knows any way to know how a given sfs was made?

Any command or mksquashfs option to get details about the block size, xdict size, compression used, of a given sfs.

[musher0]

Hi drunkjedi.

Good question. Why do you want to know? As long as it does the job...

If you try to open a sfs file with the real less in binary mode, perhaps you can see a
kind of header? (Please see attached capture.) Now, to interpret it...

You are probably thinking of something like the zipinfo utility for the zip compressor.
I searched a bit on the Web and I have not found anything like it for squash files.

IHTH

[hamoudoudou]

compression tools
To open ISOs squash files i use UEextract..
Just to remember.
User can choose low, fast, that will give different size in resulting files.
My Opinion :compression is useful for very big files, compression makes processors work and slow down install.
A Puppy ISO compressed xz in not lighter than compressed gz, the weight would be the same, compared to one kg of feathers and one kg of iron. For old computers with only one processor, that is important not to give it work to do. (inflate the air mattress)

[nic007]

xz compression will ALWAYS produce a smaller compressed file size than gzip compression, if not, you haver done something seriously wrong somewhere (unless you are trying to compress something that has already been compressed, like an ISO with already compressed content).

[drunkjedi]

Why do you want to know? As long as it does the job...

Ohh it's just me being lazy.
I have too many sfs laying around. I don't know how they were compressed, or if there's any chance to further save space.

I searched a bit on the Web and I have not found anything like it for squash files.

I did search the web before posting, didn't find any way.

Only easy way would be to make a script that will unsquash and then re-squash given bunch of SFS with those options.

[nic007]

Why do you want to know? As long as it does the job...

Ohh it's just me being lazy.
I have too many sfs laying around. I don't know how they were compressed, or if there's any chance to further save space.

I searched a bit on the Web and I have not found anything like it for squash files.

I did search the web before posting, didn't find any way.

Only easy way would be to make a script that will unsquash and then re-squash given bunch of SFS with those options.

Right-clicking on the sfs and choosing properties from the menu will give you some description of the sfs. A block size of 1 byte (if possible) will generally indicate the highest compression. I've checked my sfs's which were compressed at maximum compression and they all report block sizes of either 1 or 2 bytes.

[musher0]

@drunkjedi:

Looking for something else, I chanced on these methods
to get info about a squashed or sfs file.

The easy way is to open a console in the directory where your sfs file is, and type:

Code: Select all

unsquash -s file.sfs

Another involves the use of dd, and another still uses

Code: Select all

strings file.sfs | head

to get "some" info.

None of these methods seem able to uncover the additional parameters used to
obtain greater compression when squashing.

IHTH.