Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 21 Jan 2018, 05:00
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
How to stuff 7% more material into your sfs's.
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 2 of 2 [22 Posts]   Goto page: Previous 1, 2
Author Message
rufwoof

Joined: 24 Feb 2014
Posts: 2194

PostPosted: Mon 27 Mar 2017, 16:09    Post subject: Re: How to stuff 7% more material into your sfs's.  

musher0 wrote:
a bit longer to create, but are perhaps a bit snappier to
unpack to RAM when you load your Puppy

LZ4 compresses less tightly (larger size), but decompresses much quicker than both gzip and xz. It's a trade off between disk IO speed and processing (decompression) and ram transfer speed. LZ4 when using multi-cores can throw out data at ram transfer speeds (extremely fast).

Reading more disk IO/decompress quickly can be quicker than less disk IO (tighter compression)/slower output (decompression).

Also bear in mind that once a program has been read in once during a session, most of that will remain memory bound (cached), such that subsequent re-runs of the same program/libs will already be in memory (cache). Especially in the case of puppy that might be say 500MB uncompressed in total, running on a PC that perhaps has 2GB or ram. Rather than loading the entire pup into ram at startup (slower bootup), just loading files as and when required will load less (gparted for instance might never even be used during a session). Slower to initially load a program (such as gparted if invoked during a session) than having it already available in memory, but no difference for subsequent re-runs of the program.

Generally not loading puppy into ram at startup, using lz4 compressed sfs's is the better overall choice IMO. Maximising xz compression will tend to see a slower system by comparison whilst IO is more predominant, but thereafter the two tend to compare anyway (once cached).

As a example I run Debian frugally with a save partition (rather than a save file/folder) and comparing having all of that inside a sfs to having all of the filesystem in the save partition (non compressed) sees very little difference between the two. For me, having all of the data in the save partition leaving a empty main sfs means that I can also boot that as though it was a full install. There is a noticeable difference between booting full versus frugal however as with frugal all changes are being recorded in memory, not back on disk (I personally like frugal as you can just reboot without saving to undo all changes made during the session i.e. I like to keep the main system pristine/factory-fresh - I only ever tend to boot as though full when I apply updates, and then immediately reboot back to frugal again afterwards).
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 4797
Location: Republic of Novo Zelande

PostPosted: Tue 28 Mar 2017, 02:10    Post subject: Re: How to stuff 7% more material into your sfs's.  

rufwoof wrote:
LZ4 compresses less tightly (larger size), but decompresses much quicker than both gzip and xz.
Any chance you could suggest a good default syntax to build an sfs from a directory using LZ4? (and do you have any idea if LZ4 is understood by Slacko?)
Back to top
View user's profile Send private message 
nic007


Joined: 13 Nov 2011
Posts: 2245
Location: Cradle of Humankind

PostPosted: Mon 03 Apr 2017, 07:20    Post subject:  

Some more testing. Best xz compression achieved with: -comp xz -b 1048576 -Xbcj x86

Results compressing Racy base sfs:
gzip = 142MB
-comp xz (default setting) = 121MB
-comp xz -b 1048576 -Xdict-size 100% = 109MB
-comp xz -b 1048576 -Xbcj x86 = 105MB

Compression slowish, decompression time not affected. Good if you are only going to compress once or seldomly.
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 11495
Location: Gatineau (Qc), Canada

PostPosted: Mon 03 Apr 2017, 12:09    Post subject:  

nic007 wrote:
Some more testing. Best xz compression achieved with: -comp xz -b 1048576 -Xbcj x86

Results compressing Racy base sfs:
gzip = 142MB
-comp xz (default setting) = 121MB
-comp xz -b 1048576 -Xdict-size 100% = 109MB
-comp xz -b 1048576 -Xbcj x86 = 105MB

Compression slowish, decompression time not affected. Good if you are only going to compress once or seldomly.
Hi nic007.

Your memory came back, I see!!!

"Oh, sh?t." That's a 35% space savings over gz. Wow.

Bravo nic007.

_________________
musher0
~~~~~~~~~~
"Logical entities must not be multiplied beyond necessity." | |
« Il ne faut pas multiplier les entités logiques sans nécessité. » (Ockham)
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 4797
Location: Republic of Novo Zelande

PostPosted: Mon 03 Apr 2017, 12:52    Post subject:  

nic007 wrote:
Some more testing. Best xz compression achieved with: -comp xz -b 1048576 -Xbcj x86

Thanks for the parameters. Using this gave me the following comparison for my base sfs:

454MB using gz method

351MB using Mushero xz syntax

335MB using nic007 xz syntax

Unfortunately both of the xz methods create an issue for me when the system tries to run my firewall setup script in /root/Startup so I assume I must have a delay in unpacking something at a critical moment.

Not a big issue and probably something I can find a way around (I think it is due to the fact I use xdotool to generate a keyboard action during the firewall script. Bad practice really).

The other issue for me is that the xz methods both take 30 seconds longer than gz to boot my machine but then i am working with quite a large sfs so maybe not a problem for others.

The significant space saving does make the xz syntaxes useful for making best use of storage space.
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2194

PostPosted: Mon 03 Apr 2017, 15:26    Post subject: Re: How to stuff 7% more material into your sfs's.  

greengeek wrote:
rufwoof wrote:
LZ4 compresses less tightly (larger size), but decompresses much quicker than both gzip and xz.
Any chance you could suggest a good default syntax to build an sfs from a directory using LZ4? (and do you have any idea if LZ4 is understood by Slacko?)

I suspect its not in Slacko.

It has to be built into the kernel as well as having updated squashfs-tools ... in which mksquashfs will typically have just -comp lz4 (basic choice) or -comp lz4 -Xhc (higher compression choice (decompression speed isn't any slower, but in decompressing less due to higher compression will throughput quicker)). Compression ratio isn't good. Higher speed comes with less complexity (less compression).

mksquashfs --help will show if the squashfs-tools version you have supports lz4, but the kernel might not necessarily also support it.
Back to top
View user's profile Send private message 
greengeek


Joined: 20 Jul 2010
Posts: 4797
Location: Republic of Novo Zelande

PostPosted: Tue 04 Apr 2017, 04:34    Post subject: Re: How to stuff 7% more material into your sfs's.  

rufwoof wrote:
mksquashfs --help will show if the squashfs-tools version you have supports lz4, but the kernel might not necessarily also support it.
Thanks - looks like only gzip and xz available. cheers!
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 2 of 2 [22 Posts]   Goto page: Previous 1, 2
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0605s ][ Queries: 12 (0.0081s) ][ GZIP on ]