Partially automated remastering of sfs files

[RSH]

No need to shout, mikeb.

Just want to bring some colour to thee world.

Anyway, ever heard of anyone shouting in green?

As far as I know, I never heard of calling it "a shout" if one is using large fonts.

Only if using strictly capital characters (ABCDEFG).

How ever: did never understand that. To me its just another "stupid law" of internet behavior.

[mikeb]

Its ok...I saw green as apposed to red....

the therapy is kicking in and I feel fine now

mike

lets have a party instead

[artsown]

Smithy, I located a discussion on removing the .XLOADED file and the fix works to get rid
of the bootup mesage concerning a power fail when running without a personal Save file
after remastering. You add the line:
rm /etc/.XLOADED
to /etc/rc.d/rc.local

To make this stick I created a personal Save file temporarily instead of editing during the
remastering process. That way, I can simply use the full normal remaster script without
interruption and editing.

So this is the second addition to my original procedure (which assumes the use of a personal
Save file initially while doing modifications to puppy). I haven't yet experimented with simply
turning off compression completely in mkquashfs instead of using the version with very
light compression. Been busy with other things here.

Thanks to all for your inputs. Hope my inputs help those interested in remastering machine-
specific and personalised pups we can use without personal Save files.

Art

[mikeb]

Size and speed without any compression are similar to using tar. a 50MB save on this pentium 3 takes half a second to a second to hard drive for example.

I think the xloaded detection is in /etc/profile...its pretty pointless so you might want to remove it...if its not there its in xwin... on windows and been a while and dodgy memory.

mike

[artsown]

Mike, I tried using no compression and the total remaster time dropped from 100 to 80
seconds while the resulting squash file size increased from 221 meg to 395 meg. The
221 meg was with the minimal compression ... with full compression the file size is
about 200 meg. There is some boot speedup without compression since no decompression
is required ....a drop from about 40 seconds to 30 seconds full to no compression.

I'm working on some other things, and don't know if I'll be running more tests and accumulating
more data or not. Seems to me that users would be very interested in seeing the results
of such tests.

Anyway, thanks for nudging me into trying out the no compression option.

Art

[mikeb]

no problem ...perhaps some of the time consumed is related to data transfer rather then compression. I had 5 seconds compared 1 second... in my case making an sfs of pup_rw containing ~30mb data using full and zero compression...pentium 3 machines with decent ide drives/cache.

Another point is mksquashfs has options for which part of the system is compressed and how duplicate files are handled... though the bulk of time savings are with compressing the main data.

Was your figure with the lower compression special build of mksquashfs..I am not clear on that. If so that does sounds interesting if we can have ~40% compression with only a 20% penalty. Loading times varied only a little...it was the save at shutdown that speed became a factor.

When making sfs files I do find up to 50% faster if I use sfs without lzma (standard squashfs) as apposed to with, with a size gain of 10-20%, but in those instances speed is not essential.

Some more scientifically compile statistics would be a good idea ...I will apply myself to that

mike

[mikeb]

Ok a quick reread...are your time figures including the cp parts? Would explain the marginal gain there....

mike

[artsown]

Mike, yes and yes. The low compression refers to that special mksquashfs having low
compression ... that link I had posted. And yes, total times include copying times.

Art

[Smithy]

Just adding my remastering notes that have been gathered from the guys on the forum. Think they match up Artsown.
Removal of:
X seems to have exited uncleanly the last time you ran puppy
this is usually because of a power failure.
If not just wait 30 seconds to start
Another simple way to get around this is to just add to /etc/rc.d/rc.local:

Code: Select all

Code:	
rm  /etc/.XLOADED (Confirmed, this is the best one).

When X is started, /etc/.XLOADED is created, so if X wasn't exited properly (like the classic "fsckme" flag: on a proper shutdown it will be deleted ; if it's not deleted you know the system has crashed).
If the periodic saving is used (or if the user manually saves with snapmergepuppy), then the saving happens while X is running, thus .XLOADED is also saved to the save file (mounted on /initrd/pup_ro1), so you need to delete it or X won't automatically start on next boot.
The same happens if you backup your save file.


ANSWER:
/etc/.XLOADED is one of the very few files you need not to copy into /tmp/etc while remastering .
Otherwise the dialogue is to be found in /usr/bin/xwin script .

The do not remove flash drive message....
usr/sbin/delayedrun..Lines 116-124 comment out. Breaks Puppy badly.

[mikeb]

Ok makes sense...I was considering ONLY mksquashfs times (standard remaster and sfs save) where you would probably find a large speed difference but in your script ..cp of the whole system plus save will be taking a good chunk of the time. So I am not going mad ..just my usual confused state.

For my saves obviously mksquashfs is the only factor since nothing else is used and speed of shutdown has to be good.

Not exactly comparing apples and oranges but useful info has been brought up regardless......

mike

[Smithy]

Mike,are you saying that one could theoretically shove all the contents of puppy onto a usb stick for example, all uncompressed, and that it would boot up really fast?

I was thinking that if they were all uncompressed at least you could tweak things, and it would all be done there and then without having to squash it up again.

I'm pretty much done with isos these days and just use vmlinux, initrd.gz, puppy main sfs and the syslinux cfg and the msg and pretty picture on boot.

[mikeb]

Mike,are you saying that one could theoretically shove all the contents of puppy onto a usb stick for example, all uncompressed, and that it would boot up really fast?

no lol

I was in my case referring to how I save on puppy.

What you might be after is making a full install on a usb stick...there have been threads on the subject in the past.

Closest I have is a frugal save file containing the full system...so like a full in a frugal buit thats aimed at fat32/ntfs partitions.

mike

[Smithy]

What you might be after is making a full install on a usb stick...there have been threads on the subject in the past.

Maybe, but the post highway could be littered with puppy roadkill of abandoned experiments

It took me weeks to wonder why the sticks wouldn't boot puppy until a good old rcrn51 howto produced the magic code:

2. Plug in the flash drive but don't mount it. Type:

Code: Select all

Code:
syslinux /dev/sdxy

So I sort of stick to how Barry K designed it to be used (I think).
Like you said, there was some good stuff brought up on remastering from those at the coalface. I'm handing out the canaries at the lift shaft

[mikeb]

Hmm i would guess the method was to make the installer think the flash stick was a hard drive.

That info is gained from /etc/rc.d/PUPSTATE where usb and hard drives are listed separately ...moving over or adding the flash drive might be all thats needed the follow the hard drive procedure.

syslinux for booting or I usually use grub4dos with bootlace.com /dev/sd(x)

a full on a stick is probably less of a problem than pupmode 13 and speed wise few of use still have only usb1 around...I actually did run XP from an SD card on a netbook...was fine so I cannot see puppy being a problem.

mike

[Smithy]

Yes, I would think puppy would thrash much less than xp.
Did you turn off virtual memory in xp when you ran from sd card?

I know you are a fan of win2k, a guy once told me you could run a city on win2k.

Maybe that mayor of New York pressed a button to close the bridges and caused traffic chaos.

[mikeb]

Did you turn off virtual memory in xp when you ran from sd card?

yes.... seemed a good idea...the netbook had 1GB ram so was plenty without anyway.

Only time it got slow was adding/detecting a device driver....but thats a slow tedious process on windows anyway.

mike