Question about unsquashfs/mksquashfs
Question about unsquashfs/mksquashfs
Hi all,
Have a question that has confused me for a long time, and I'm sure there is a simple answer (or something I'm not understanding) but I haven't thought of it or come across it yet.
The question is thus: any main puppy_save_####.sfs file.
Why is it, if you take any main puppy_save.sfs (from any pup ISO), and just do the following, you'll end with quite significant different size .sfs files:
1) Example: click on puppy-slacko_5.7 ISO to open it in Rox.
2) Note the exact MB size of the puppy_slacko_5.7.sfs that is made with compression "xz"
3) copy the puppy_slacko_5.7.sfs to an empty folder on any partition
3) open a terminal in that folder and run 'unsquashfs puppy_slacko_5.7.sfs'
4) see the "squashfs-root" folder it makes
5) turn right around, in same terminal, and run: 'mksquashfs squashfs-root/. 'new.fs' -all-root -noappend -comp xz"
6) then note the MB size of the this new.sfs
Should it NOT be the exact same size as when you started? I have run this test on at least 12-15 (and more, given the number of 'frugal' installs I've run over the years) and every single time, your 'new.sfs' will be significantly larger than the original .sfs you started with from the ISO. And I am talking anywhere from 10% to sometimes 20% larger.
Question is: Why? Why is not the exact same sized compressed xz .sfs file made??
Is there something in my string of terminal command appendages that I am running that I shouldn't, or that maybe I am not running/adding and I should?
Thanks for any help/feedback about not understanding this (please note, if it helps, it hasn't mattered if the hard drive/ssd/usb partition(s) have been formatted ext 2/3/4, the results are still the same).
[Edit---in writing the message, simply forgot the "." after the "squashfs-root-/" .........see Greengeek's comment below]
Have a question that has confused me for a long time, and I'm sure there is a simple answer (or something I'm not understanding) but I haven't thought of it or come across it yet.
The question is thus: any main puppy_save_####.sfs file.
Why is it, if you take any main puppy_save.sfs (from any pup ISO), and just do the following, you'll end with quite significant different size .sfs files:
1) Example: click on puppy-slacko_5.7 ISO to open it in Rox.
2) Note the exact MB size of the puppy_slacko_5.7.sfs that is made with compression "xz"
3) copy the puppy_slacko_5.7.sfs to an empty folder on any partition
3) open a terminal in that folder and run 'unsquashfs puppy_slacko_5.7.sfs'
4) see the "squashfs-root" folder it makes
5) turn right around, in same terminal, and run: 'mksquashfs squashfs-root/. 'new.fs' -all-root -noappend -comp xz"
6) then note the MB size of the this new.sfs
Should it NOT be the exact same size as when you started? I have run this test on at least 12-15 (and more, given the number of 'frugal' installs I've run over the years) and every single time, your 'new.sfs' will be significantly larger than the original .sfs you started with from the ISO. And I am talking anywhere from 10% to sometimes 20% larger.
Question is: Why? Why is not the exact same sized compressed xz .sfs file made??
Is there something in my string of terminal command appendages that I am running that I shouldn't, or that maybe I am not running/adding and I should?
Thanks for any help/feedback about not understanding this (please note, if it helps, it hasn't mattered if the hard drive/ssd/usb partition(s) have been formatted ext 2/3/4, the results are still the same).
[Edit---in writing the message, simply forgot the "." after the "squashfs-root-/" .........see Greengeek's comment below]
Last edited by belham2 on Thu 28 Sep 2017, 08:24, edited 1 time in total.
Re: Question about unsquashfs/mksquashfs
I can't offer any info re xz compression - but is it just a typo that the "." is missing from "squashfs-root/." or does that make some sort of difference?belham2 wrote:5) turn right around, in same terminal, and run: 'mksquashfs squashfs-root/ 'new.fs' -all-root -noappend -comp xz"
Re: Question about unsquashfs/mksquashfs
greengeek wrote:I can't offer any info re xz compression - but is it just a typo that the "." is missing from "squashfs-root/." or does that make some sort of difference?belham2 wrote:5) turn right around, in same terminal, and run: 'mksquashfs squashfs-root/ 'new.fs' -all-root -noappend -comp xz"
Hi Greengeek,
Fingers were a flying & typing, and the brain was not keeping up. Translation: just simply forgot it
Hi belham2
To get information only about original sfs you can use the -s option
And it may be something like this
So, for example, to set the same block size for the new sfs
Still maybe not the exact same size, could depend on other options too.
EDIT: Also added "-Xbcj x86" to above example, don't know if it makes any difference in size though.
For info: blocksize can be set also to 1M (double of 524288) , which gives more compression, smaller file (but booting and running will probably be a little slower then)
Fred
To get information only about original sfs you can use the -s option
Code: Select all
unsquashfs -s puppy_slacko_5.7.sfs
Code: Select all
Compression xz
Dictionary size 524288
Filters selected: x86
Block size 524288
Code: Select all
mksquashfs squashfs-root new.sfs -comp xz -b 524288 -Xbcj x86
EDIT: Also added "-Xbcj x86" to above example, don't know if it makes any difference in size though.
For info: blocksize can be set also to 1M (double of 524288) , which gives more compression, smaller file (but booting and running will probably be a little slower then)
Fred
Last edited by fredx181 on Thu 28 Sep 2017, 09:21, edited 3 times in total.
Re: Question about unsquashfs/mksquashfs
And presumably that applies to the "new.fs" spelling as well? (If I use that syntax I do get a larger file created...)belham2 wrote:5) turn right around, in same terminal, and run: 'mksquashfs squashfs-root/ 'new.fs' -all-root -noappend -comp xz"
Do you get the same results if you start with an xz compressed sfs that you created yourself? If not then maybe the pup developer created the original pup with different xz parameters (or more efficient xz utility) than is available to you?
@greengeek-------you're killing me here. Did you (or still do?) proof Queen Mum's releases after she did them, and busted her chops for any digressions?
@Fred----Thanks! Very useful info, I am trying it with a few pups to see if it makes it better and/or a difference.
P.S. My name is 'Belham2', and I hereby absolve myself of all spelling, punctuation, grammar, mis-typo errorssssssss----hahahha--- along with any idiocy I concoct, in this thread and/or messages. This shall also apply to all future threads/posts.
@Fred----Thanks! Very useful info, I am trying it with a few pups to see if it makes it better and/or a difference.
P.S. My name is 'Belham2', and I hereby absolve myself of all spelling, punctuation, grammar, mis-typo errorssssssss----hahahha--- along with any idiocy I concoct, in this thread and/or messages. This shall also apply to all future threads/posts.
xz has different compression settings, you need to use the exact same options to get the same compressed size. For example: when using the default xz compression setting, I can compress the Racy sfs to 120 MB. When using the maximum setting, I can compress it to 105MB. Pre-kernel 3 puppys do not support xz compression.
Hi nic007!nic007 wrote:xz has different compression settings, you need to use the exact same options to get the same compressed size. For example: when using the default xz compression setting, I can compress the Racy sfs to 120 MB. When using the maximum setting, I can compress it to 105MB. Pre-kernel 3 puppys do not support xz compression.
Ok, that is totally new to me (as was Fred's post).
Can I ask: is there a way (in terminal or otherwise) I can find out what compression was used in a released puppy's ISO main .sfs file? I mean, when ya click on the ISO, you can see the puupy_#####.sfs directly, but is there anyway to know what was used "cz" -wise? Should I just assume it was the max? And if that is true, why is not my setting (as I entered it above in earlier message) not using the max?? What has to happen in the terminal commands for max to be used? Something like what Fred wrote?:
Code: Select all
mksquashfs squashfs-root new.sfs -comp xz -b 524288 -Xbcj x86
Code: Select all
mksquashfs squashfs-root new.sfs -comp xz -b 1048576 -Xbcj x86
What the heck is "max" when using 'xz'? Can I just write:
Code: Select all
mksquashfs squashfs-root new.sfs -comp xz -b maximum (or max) -Xbcj x86
What is confusing me is that I did like Fred suggested above in his post
Code: Select all
unsquashfs -s puppy_slacko_5.7.sfs
Ok, wanted to followup in case anyone else was curious about this thread, and wondered how to control "xz" compression used via the terminal.
This site explains it clearly;
https://www.rootusers.com/13-simple-xz-examples/
Basically, you are looking at using a number system (between '-0' and '-9'. The level of compression applied to a file using xz can be specified as a value between 0 (less compression) and 9 (best compression). Default running 'xz' in terminal with no flags uses the default of "6".
There is also an "extreme mode" that can be utilized. The -e or --extreme flag can be specified to use more CPU when encoding to increase the compression ratio, however this will take more time. And there is also a -T or --threads flag where you can specify the number of worker cpu core threads to use, potentially increasing xz performance.
Thanks again t all who replied here. I am successfully now doing unsqaushfs and mksquashfs that is governed by the media (size) I am installing too, and also whether I want to consider that media's speed perfromance based on the 'xz' compression used.
This site explains it clearly;
https://www.rootusers.com/13-simple-xz-examples/
Basically, you are looking at using a number system (between '-0' and '-9'. The level of compression applied to a file using xz can be specified as a value between 0 (less compression) and 9 (best compression). Default running 'xz' in terminal with no flags uses the default of "6".
There is also an "extreme mode" that can be utilized. The -e or --extreme flag can be specified to use more CPU when encoding to increase the compression ratio, however this will take more time. And there is also a -T or --threads flag where you can specify the number of worker cpu core threads to use, potentially increasing xz performance.
Thanks again t all who replied here. I am successfully now doing unsqaushfs and mksquashfs that is governed by the media (size) I am installing too, and also whether I want to consider that media's speed perfromance based on the 'xz' compression used.
In a terminal run mksquashfs and it will show the compression choices it supports. Those might not match what is supported by the kernel i.e. if you make a puppy.sfs using a compression method that the kernel doesn't support then it wont boot.
In the absence of both mksquashfs and kernel support for lz4 I usually go with lzo level 1 compression myself for throughput (speed). Larger filesize but quick to decompress (and compress). For tightest compression (smallest file) a high compression xz tends to do best, but throughput is slower.
mksquashfs somedir somefile.sfs -comp lzo -Xcompression-level 1
In the absence of both mksquashfs and kernel support for lz4 I usually go with lzo level 1 compression myself for throughput (speed). Larger filesize but quick to decompress (and compress). For tightest compression (smallest file) a high compression xz tends to do best, but throughput is slower.
mksquashfs somedir somefile.sfs -comp lzo -Xcompression-level 1
I was curious about that also and found that using: -Xbcj x86 , makes smaller size, so got the same size as original (having blocksize 524288) using:belham2 wrote:What is confusing me is that I did like Fred suggested above in his post
Code:
unsquashfs -s puppy_slacko_5.7.sfs
I found the exact that was used. Yet despite entering that exact same in terminal, and mksquashfs-ing, the puppy-slacko_5.7.sfs came out a "BIGGER' size than what was in the ISO---despite nothing being added to the .sfs. So...... Confused.....I'm lost at what I'm doing wrong.
Code: Select all
mksquashfs squashfs-root new.sfs -comp xz -b 524288 -Xbcj x86
Code: Select all
mksquashfs squashfs-root new.sfs -comp xz -b 524288 -Xdict-size 524288 -Xbcj x86
Fred
Check this thread: http://www.murga-linux.com/puppy/viewtopic.php?t=110035
-
- Posts: 43
- Joined: Tue 20 Dec 2016, 23:16
-
- Posts: 328
- Joined: Wed 25 Jun 2014, 20:31
I haven't found any either, but you can mount the SFS file (by clicking on it in ROX Filer; click again later to unmount) and then check the size of the directory where it is mounted.Jose A. Senna wrote:Is there an option to make unsquashfs display the size the file(s) shall have after decompression? I could not find it with unsquashfs --help
For example, I've mounted my devx.sfs. The mount point is /mnt/+root+wary_devx_511.sfs. When I right-click that directory and use ROX's "Count" option, it tells me that the disk usage is 435MB, while choosing "Properties" tells me the size of the compressed file, 106MB. You can also run that folder through a graphical disk mapping tool like GdMap, or use a command like du in a terminal:
Code: Select all
# du -sbh /mnt/+root+wary_devx_511.sfs
435M /mnt/+root+wary_devx_511.sfs
This will tell you the SFS content's actual size, which might be different from the space that it takes up on disk when it's uncompressed. When I actually unsquash that SFS, "Count" still says the resulting folder is 435MB big, but "Properties", which seems to show the size on disk, is now up to 493MB. (Difference between actual file size and size on disk.)
So it seems that the above file size checks can give you a basic idea of how big the SFS's contents may be, but you won't know exactly how much disk space they will occupy until you unsquash them.
-
- Posts: 43
- Joined: Tue 20 Dec 2016, 23:16