Replace your savefile/folder with an sfs storage file

[nic007]

Tested a few Puppies I had installed with test savefiles I had for another purpose, to be able to quickly test.

For newer adrv pups, Tahr 6.0.5 and Slacko 6.3.2 seemed to work OK, but noticed a bit of slowness. Not sure if sign of problems to come. Xenial 7.0.3 partially worked on the first run, but then after the 2nd run started to fail with missing panel and menus. I think something is corrupted.

For older pups with the zdrv swap, Precise 5.7.1 always reverts to Xorg Wizard. Slacko 5.7.0 seems to work the best, seems normal. Racy 5.5 worked but required a few workarounds in the script.

I think one major thing is the danger of running this script on VFAT/Fat32 frugal install. And not sure how much NTFS is fully compliant. I made a change in the Racy test to use a Linux filesystem instead of the vfat on /mnt/home, and then it seemed to work OK.

Probably a lot to ask of a single script to account for all situations and pup versions. Even individual Puppies have many workarounds in the save scripts like snapmergepuppy, etc. And even now Quirky 8.1.6 frugal persistence is broken, so it's par for the course.

Your working puppyfilesystem must always be in a linux environment (partition) or RAM if you have enough. Works perfectly for me as is in Racy5.5 and Tahr605. If you have problems with the script, it will be best to do a proper remaster first and then use the script furtheron. This is not a remaster script. It's impossible for me to know the condition of your savefile. BTW- It's possible to do this in a non-linux partition by creating a linux environment within.

[jd7654]

I think it's helpful to show the basic method, but maybe some caveats can be given like expected filesystem type, etc. Many Puppy frugal installs on Windows partitions.(like mine are)

Also, just wondering about the differences in the new and old scripts for the file/directory exclusions:

diff:
< rm -r ./tmp ./mnt ./initrd ./sys ./root/.cache/* ./root/.thumbnails/* ./root/.Trash/* ./root/.XLOADED ./var/log
> rm -r ./tmp ./initrd ./mnt ./sys ./root/.Trash/* ./root/.cache/* ./etc/.XLOADED ./var/log

Was that tuned specifically for Tahr 6.0.5 and Racy 5.5? I did seem to work for Racy 5.5, but I did have to change ./etc/.XLOADED to ./root/.XLOADED for Slacko 5.7.0 to not have the X improper shutdown warning.

I also tried again with the most problem test with Xenial 7.0.3, and moved it from NTFS to Ext4 partition. Same problems with menu and panel disappearing on 2nd run. Something is causing a problem for it being moved into sfs.

BTW, I'm using a slightly modified version of the script with variables to be more generic so can test across different versions, and added mount so that it works on 2nd run also when no savefile or /mnt/home is existing. Works OK with Tahr and Slacko 6.3.2:

Code: Select all

#!/bin/sh
. /etc/DISTRO_SPECS
. /etc/rc.d/PUPSTATE
PUPSFSFILE=`echo $DISTRO_ADRVSFS | tr -d "'"`
PUPSFSDIR=`echo $PSUBDIR | tr -d "'"`
PUPSFSHD=`echo $PDEV1 | tr -d "'"`

mkdir /mnt/home
mount /dev/$PUPSFSHD /mnt/home
mkdir /mnt/home/puppyfilesystem
cp -a /initrd/pup_a/* /initrd/pup_ro1/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
wait
cd /mnt/home/puppyfilesystem
rm -r ./tmp ./mnt ./initrd ./sys ./root/.cache/* ./root/.thumbnails/* ./root/.Trash/* ./root/.XLOADED ./var/log
wait
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Press OK when ready" 0 0
rm /mnt/home$PUPSFSDIR/$PUPSFSFILE
mksquashfs /mnt/home/puppyfilesystem /mnt/home$PUPSFSDIR/$PUPSFSFILE
rm -r /mnt/home/puppyfilesystem
wait
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000

exit

Edit: corrected wrong code posted, old puppy code versus new puppy code, 3 lines different.

[nic007]

I think it's helpful to show the basic method, but maybe some caveats can be given like expected filesystem type, etc. Many Puppy frugal installs on Windows partitions.(like mine are)

Also, just wondering about the differences in the new and old scripts for the file/directory exclusions:

diff:
< rm -r ./tmp ./mnt ./initrd ./sys ./root/.cache/* ./root/.thumbnails/* ./root/.Trash/* ./root/.XLOADED ./var/log
> rm -r ./tmp ./initrd ./mnt ./sys ./root/.Trash/* ./root/.cache/* ./etc/.XLOADED ./var/log

Was that tuned specifically for Tahr 6.0.5 and Racy 5.5? I did seem to work for Racy 5.5, but I did have to change ./etc/.XLOADED to ./root/.XLOADED for Slacko 5.7.0 to not have the X improper shutdown warning.

I also tried again with the most problem test with Xenial 7.0.3, and moved it from NTFS to Ext4 partition. Same problems with menu and panel disappearing on 2nd run. Something is causing a problem for it being moved into sfs.

BTW, I'm using a slightly modified version of the script with variables to be more generic so can test across different versions, and added mount so that it works on 2nd run also when no savefile or /mnt/home is existing. Works OK with Tahr and Slacko 6.3.2:

Code: Select all

#!/bin/sh
. /etc/DISTRO_SPECS
. /etc/rc.d/PUPSTATE
PUPSFSFILE=`echo $DISTRO_PUPPYSFS | tr -d "'"`
PUPSFSDIR=`echo $PSUBDIR | tr -d "'"`
PUPSFSHD=`echo $PDEV1 | tr -d "'"`

mkdir /mnt/home
mount /dev/$PUPSFSHD /mnt/home

mkdir /mnt/home/puppyfilesystem
cp -a /initrd/pup_ro2/* /initrd/pup_ro1/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
wait
cd /mnt/home/puppyfilesystem
rm -r ./tmp ./initrd ./mnt ./sys ./root/.Trash/* ./root/.cache/* ./root/.XLOADED ./var/log 
wait
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Press OK when ready" 0 0
rm /mnt/home$PUPSFSDIR/$PUPSFSFILE
mksquashfs /mnt/home/puppyfilesystem /mnt/home$PUPSFSDIR/$PUPSFSFILE
rm -r /mnt/home/puppyfilesystem
wait
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000

exit

Yes, XLOADED is in /etc for older puppys and in /root for newer puppy's it seems. The script is an example and can be used as a base for your own ideas, etc. I hope it is useful in that regard. I only tested it with the puppys I use Racy and Tahr

[jd7654]

Yes, XLOADED is in /etc for older puppys and in /root for newer puppy's it seems. The script is an example and can be used as a base for your own ideas, etc. I hope it is useful in that regard. I only tested it with the puppys I use Racy and Tahr

Yes it was useful to learn some new tricks. Thanks.
But I'm figuring out the persistence is a bit more complicated than I thought. Looking at some of Barry's and others various save scripts, there is a fair amount of workarounds to make it work on a live system.

Currently, I'm still just using a (closed) save file dumped into an sfs file, done offline, which seems to work fine for my purposes previously stated. I can update by dumping back but that is a lot of steps. Would be nice if Puppy had the option for sfs save storage like Quirky frugal does. That would be ideal.

[nic007]

Yes, XLOADED is in /etc for older puppys and in /root for newer puppy's it seems. The script is an example and can be used as a base for your own ideas, etc. I hope it is useful in that regard. I only tested it with the puppys I use Racy and Tahr

Yes it was useful to learn some new tricks. Thanks.
But I'm figuring out the persistence is a bit more complicated than I thought. Looking at some of Barry's and others various save scripts, there is a fair amount of workarounds to make it work on a live system.

Currently, I'm still just using a (closed) save file dumped into an sfs file, done offline, which seems to work fine for my purposes previously stated. I can update by dumping back but that is a lot of steps. Would be nice if Puppy had the option for sfs save storage like Quirky frugal does. That would be ideal.

But that is exactly what I tried to achieve here and it works for me. Are you following the instructions correctly? What are your main issues which you encounter with this method? How does Quirky's method work?

[jd7654]

But that is exactly what I tried to achieve here and it works for me. Are you following the instructions correctly? What are your main issues which you encounter with this method? How does Quirky's method work?

The procedure works generally, but there were small issues that are not apparent with a standard savefile. Tiny little bugs, which makes me wonder what else is possibly missing or corrupted.

The weird thing with Xenial 7.0.3, was that after 2nd sfs save, there were two cpu temp icons in the tray, and then the right click menu and panel menu lost all but one of its entries, but then most came back. Not acting normally at all. But that is and old version maybe later I can try with current Xenial.

But I also moved Tahr 6.0.5 and Racy 5.5 to the same Ext4 partition, and I agree with your results, they appear to be working normally even after several sfs saves and reboots, as far as I can tell.

I tried another test, moved Precise 5.7.1 over from an NTFS partition to an Ext4 partition. Same problem as described above. After 2nd sfs save and reboot, it came up in Xorg video wizard, Mind you, it booted fine straight to the desktop on the new default install, never had to go through the Xorg video wizard at all. Then I just set first run settings, got IP, browsed a couple sites, and saved the savefile. That was only thing in save file. Rebooted with save, and then ran the sfs save script. So it appears something was lost in the sfs save that caused it to spawn the Xorg video wizard on reboot. Not sure what that is.

That was just using the same script as yours basically, with just the changes for directory variables and operating on old puppy:

Code: Select all

#!/bin/sh
. /etc/DISTRO_SPECS
. /etc/rc.d/PUPSTATE
PUPSFSFILE=`echo $DISTRO_PUPPYSFS | tr -d "'"`
PUPSFSDIR=`echo $PSUBDIR | tr -d "'"`
PUPSFSHD=`echo $PDEV1 | tr -d "'"`

mkdir /mnt/home
mount /dev/$PUPSFSHD /mnt/home
mkdir /mnt/home/puppyfilesystem
cp -a /initrd/pup_ro2/* /initrd/pup_ro1/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
wait
cd /mnt/home/puppyfilesystem
rm -r ./tmp ./initrd ./mnt ./sys ./root/.Trash/* ./root/.cache/* ./root/.XLOADED ./var/log 
wait
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Press OK when ready" 0 0
rm /mnt/home$PUPSFSDIR/$PUPSFSFILE
mksquashfs /mnt/home/puppyfilesystem /mnt/home$PUPSFSDIR/$PUPSFSFILE
rm -r /mnt/home/puppyfilesystem
wait
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000

exit

Edit: Whoops, looks like I posted the wrong code earlier, that was the old Puppy code. Should have had new Puppy code with 3 lines changed including:
PUPSFSFILE=`echo $DISTRO_ADRVSFS | tr -d "'"`

[nic007]

Tried with Precise Retro. There seems to be a display driver conflict/bug/issue with this distribution. As you mentioned, works first time and then problems. This does not have anything to do with the script though. I've had these issues with Precise before running in RAM. As for menus - sometimes JWM does not update its menus correctly. Run the command fixmenus in terminal and restart JWM afterwards.

[jd7654]

Tried with Precise Retro. There seems to be a display driver conflict/bug/issue with this distribution. As you mentioned, works first time and then problems. This does not have anything to do with the script though. I've had these issues with Precise before running in RAM. As for menus - sometimes JWM does not update its menus correctly. Run the command fixmenus in terminal and restart JWM afterwards.

Well, that's not *precisely* what I was experiencing. I'm not having any general issues with Precise. It has been running fine on a savefile: make changes, reboot, no problem, changes are saved and it works, over and over again

Problems were only introduced with the change from extfs savefile to this sfs file method. Something was lost in the conversion. Whatever the underlying fault is, if the behavior is different and buggier on the new sfs method, then I'm not inclined to use it. No worries though, I'm just playing around with it.

[nic007]

There is definitely an issue with Precise (running without a savefile). Admittedly, I haven't used it for long periods before but I'm aware of this popup screen asking to choose between intel driver and vesa or whatever, something I never experienced before with other puppys. I've also had standby problems with Precise using my laptop. As for Xenial - as far as I know this is still a work in progress so odd behaviour can be expected. Anyways - works for me with the puppys I'm using, have been doing it this way for months.

[jd7654]

Yeah, I don't know what the issue could be, could be something small. Especially since you've been running for months, it's probably all good.

Even with Precise, the way I'm running it, with the whole save file dumped into an sfs file, offline. and then doing the zdrv swap, that seems to work fine. There's something different about the copying of files from the live filesystem.

As for Quirky, Barry's script is quite big as it handles a lot of stuff and situation, but it seemed to work great in 8.0.(but broken in 8.1.6)
I think this is the main part that saves persistence into s.sfs file:

[nic007]

I have this problem with Precise running the original puppy files as contained in the official distro and without a savefile. Just double-checked again, same story. We are not talking of anything saved here, totally running in RAM as fresh install. Could be a hardware conflict but as I said, the only puppy I have ever used to do this. Sometimes it will bootup to desktop without a hick, so don't know it's a mystery. Then when I actually use may method and save the settings from a savefile it works on reboot (with this saving sfs loaded and no savefile) just to display the same peculiar behaviour at a later reboot (without having changed anything whatsoever since). As a matter of interest, how big is your "off-line" sfs file containing your savings and how do you load this file?

[jd7654]

OK, I figured out the problem with Precise 5.7.1:

The Xorg video wizard pops up every time if pfix=fsck and no savefile exists.

In my other testing where manually I dumped the entire savefile into a sfs file, I was coming up with pfix=ram so it went straight to desktop and no Xorg video wizard ever appeared. I then later created a savefile which also prevents the Xorg video wizard from appearing again.

So I tried the sfs conversion method again, and with specified pfix=ram it does come up straight to fully configured desktop. So yes, appears there is a distro specific issue with the sfs replacement of savefile, but just a boot option change.

[jd7654]

Also, still curious about the directory exclusions you had for both scripts:

diff: (new and old puppy versions)
< rm -r ./tmp ./mnt ./initrd ./sys ./root/.cache/* ./root/.thumbnails/* ./root/.Trash/* ./root/.XLOADED ./var/log
> rm -r ./tmp ./initrd ./mnt ./sys ./root/.Trash/* ./root/.cache/* ./etc/.XLOADED ./var/log

Were those directory exclusions based on a document or file, or from your own preferences, or trial and error? I understand the .XLOADED part.

[nic007]

/tmp, /initrd, /mnt, /sys, .XLOADED are deleted to avoid bootup and configuration problems. The others are optional personal preference eg. I delete the contents of my browser cache and trash.

[jd7654]

/tmp, /initrd, /mnt, /sys, .XLOADED are deleted to avoid bootup and configuration problems. The others are optional personal preference eg. I delete the contents of my browser cache and trash.

OK, thanks for confirming.

I'm still seeing a few tiny errors here and there after sfs saves, even with Tahr and Racy. Little things like missing changes or file duplicates, which may explain some of the strange anomalies I saw in Xenial with duplicate CPU temp icon and menu corruption.

For now I think I'll just stick with my savefile dump to sfs method. But I don't see why the general process you've detailed for savefile to sfs conversion shouldn't work, just maybe needs more detail and testing.

I appreciate the effort and testing of the Puppy developers to make sure each of their releases has proper save file functionality. So further improvement might involve incorporating in whole or in part the savefile operations.

Have you looked at the Puppy script for savefile, like the file I mentioned earlier: /usr/sbin/snapmergepuppy #and the comments
There's quite a bit of intricacy in the file/directory exclusion, and also considerations for layers, states and the copy operation itself in the rest of the script.

Sample file inclusion/exclusion for Racy:

Code: Select all

find . -mount -not \( -regex '.*/\.wh\.[^/]*' -type f \) -not -type d |  sed -e 's/\.\///' | grep -v -E '^mnt|^initrd|^proc|^sys|^tmp|^pup_|^zdrv_|^root/tmp|_zdrv_|^dev/\.|^dev/fd|^dev/pts|^dev/shm|^\.wh\.|^var/run|^root/ftpd|^var/tmp|\.XLOADED$' | grep -v -E -i '\.thumbnails|\.trash|trash/|\.part$'  |

And for Tahr:

Code: Select all

find . -mount \
	   -not -path . \
	   -not -type d \
	   -regextype posix-extended \
	   -not \( -regex '.*/\.wh\.[^\]*' -type f \) \
	   -not \( -regex '^./mnt.*|^./initrd.*|^./proc.*|^./sys.*|^./tmp.*|^./pup_.*|^./zdrv_.*|^./root/tmp.*|.*_zdrv_.*|^./dev/\..*|^./dev/fd.*|^./dev/pts.*|^./dev/snd.*|^./dev/shm.*|^./dev/tty.*|^./\.wh\..*|^./var/run.*|^./root/ftpd.*|^./var/tmp.*|.*\.XLOADED$' \) \
	   -not \( -regex '.*\.thumbnails.*|.*\.part$|.*\.crdownload$' \) \
	   -printf "%s %C@ %P\n" |

[nic007]

I go for small and very basic coding as long as it works for me. In my years with puppy, I've learnt to keep things simple. Large code texts gives me a headache and is of no interest to me really. In the end you do what's best for yourself, that's the beauty of puppy..fiddle with it till it suits your personal needs. Don't know why you have problems with saving the changes to a file you have edited, works for me (the file must obviously be part of the running filesystem). The only thing that does not save for me is changes to sound levels. This is because changes to the sound state is not saved to /initrd/pup_rw during a session but only at shutdown by running the shutdown script.

[jd7654]

Well, only thing I can say in general is:

Just be open to the idea that reducing hundreds of lines of code to just a few lines could possibly reduce functionality. And that extra code was put there for a reason, so removing it could have unintended consequences.

That's all I have to say about that.

Cheers

[nic007]

Well, only thing I can say in general is:

Just be open to the idea that reducing hundreds of lines of code to just a few lines could possibly reduce functionality. And that extra code was put there for a reason, so removing it could have unintended consequences.

That's all I have to say about that.

Cheers

I agree with you but if a simple method works for me why should I bother further? You are free to produce the perfect solution for all puppys and all situations and share it with the puppy community...and I encourage you to do so. I'm sure the puppy community will embrace and appreciate it. It does seem as if you have some coding skills (most probably much better than my efforts which are very basic). Looking forward to see and test your efforts. Go for it!

[jd7654]

No, thanks. I like to eat the sausage, but don't necessarily want to get involved with how it is made.

I was just curious and poking around here with this sfs method. Did some scripting a long time ago, but am a piss-poor coder. I can see now looking at it and tinkering some more that there is more complexity there in the layers and the "whiteouts" files that I was seeing issues with, something the savefile script handles.

[nic007]

Deleted