Keep your savefile slim and healthy

How to do things, solutions, recipes, tutorials
Message
Author
User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#121 Post by mikeb »

Multisession [Multi-session live-CD/DVD] can save to dated save folders. That's why it's called "multisession".
perhaps my wording was not clear enough..'and have a not save option' might be more clear.

In other words it truly loads all to ram including saves (sfs part only if enough ram) ..... and there is no persistent save file on a hard drive/usb. It inherently, and as part of the system has a no save option. My point was usb DOES have a save file which is mounted in use...just has the extra tmpfs for session changes which then gets merged. Hence the inability to remove the usb flash stick. The impression was given that puppy loads to ram saves and all regardless of mode...which is not the case EXCEPT for multisession. PUPMODE=13 was designed reduce save file writes rather than be independent of one.

Another thing mentioned was decompressing of sfs.... not so...only the required data read is decompressed on the fly when needed but the sfs remains the same size sat in another tmpfs.

Thing is if talking about the system and optimising then it needs to be clear what the system is including my wording of it :)
Its important to know what goes where....

There is a nice set of diagrams of the various modes somewhere.... a picture definitely paints a thousand words.


As an aside the sfs save method I made is based on multisession's total ram approach but usable on flash and hard drives. In other words achieve what all these hacks with PUPMODE=13 are trying to do...ie be free of ANY drives once booted and discard all at the end if desired.

mike

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#122 Post by rufwoof »

Doesn't pupmode 5 also have a 'not save' option?

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#123 Post by mikeb »

well I suppose so.... thats meant to be the first run mode before choosing a save option so I suppose its intended as a one off..... some use it as a ram only no save option.

sfs save basically runs in pupmode=5....

I feel hairs getting split :D

I was just referring to 'official' puppy usage rather than the wild and wonderful methods many here use ;)

mike

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#124 Post by rufwoof »

Just mentioned it because I run pupmode 5 all of the time and modified the shutdown code to skip asking whether to create a savefile or not (defaults to not) i.e. I run with no savefile and just remaster any changes I want preserved across reboots (increasingly infrequently). I now have all of the sfs's that I use regularly saved inside the puppy sfs (and I drop puppy sfs into initrd), so initrd holds everything and is loaded into ram at bootup. I've dropped using a swap partition as well, so nothing touches the disk at all - except a remaster if I use the HDD as a temp storage area (or if I intentionally mount a drive).

Can't recall the exact detail without looking, but seem to remember seeing/reading a suggestion of switching from pupmode 5 to pupmode 12 I think it was - as part of shutdown to equally not save. (I think it was because that would be assumed to already have been saved - or something like that ??? Or could have been pupmode 13 ???).

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#125 Post by mikeb »

wouldn't it be nice to have saved configs and data and changes all in ram with option to not save whenever you want and all done automatically and with no need for a symlinks or other such methods ...

oh it is.... ;)

yes mode 12 would do nothing at shutdown.

PUPMODE=13 should have been dropped / replaced many moons ago.

You have to break away from puppies ways... its essential.

Mike

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#126 Post by MochiMoppel »

rufwoof wrote:Can't recall the exact detail without looking, but seem to remember seeing/reading a suggestion of switching from pupmode 5 to pupmode 12
This one?

Basically all you need is

Code: Select all

#!/bin/sh
sed -i 's|PUPMODE=[0-9]*|PUPMODE=12|' /etc/rc.d/PUPSTATE
exec wmpoweroff

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#127 Post by rufwoof »

That's the one - thanks.

You're board searching abilities are clearly a lot better than mine. I more often resort to using Google rather than the local search 'feature'.

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#128 Post by mikeb »

You're board searching abilities are clearly a lot better than mine. I more often resort to using Google rather than the local search 'feature'.
try using AND with multiple queries...seems to help.
Only problem with google based searches is it can bring up really old and often obsolete information.

mike

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

Clarification

#129 Post by mikeslr »

mikeb wrote:wouldn't it be nice to have saved configs and data and changes all in ram with option to not save whenever you want and all done automatically and with no need for a symlinks or other such methods ...

oh it is.... ;)

yes mode 12 would do nothing at shutdown.

PUPMODE=13 should have been dropped / replaced many moons ago.


Mike
I think you've either (a) mis-spoke, reversing the roles of PUPMODE's 12 and 13; or (b) intending to continue the sarcasm of the first part of the quoted text, may have left the reader confused.

If you are serious that some PUPMODE should have been dropped, than it should be 12; and I'd agree with you.

The attached screenshot is from Lupu 5.28, playdaz's "original", now about 4 years old. Its a frugal install to a hard drive. I don't know if older Pups can be configured using jpeps boot parameter modification, [pmedia=ataflash] or whether their SaveConfiguration can be set to ONLY "ASK AT SHUTDOWN."

SaveConfiguration settings are reached via: Menu>System>Puppy Event Manager; then clicking the Save Session tab.

On my setup a Save Icon appears on the desktop [to enable manual Saves]; and at Shutdown a dialog appears offering the choice to Save or No Save. I'll discuss the effects of No Save below.
mikeb wrote:
Pups operate entirely in RAM.
This is only partially true...indeed addition sfs are not necessarily loaded to ram at all.
Correct, Also discussed further below.
For usb the current session is in ram but the save file is not... the ram session is saved at shutdown and periodically... seems like this is what you are talking about and in this case you can opt to not save iff you add an addon script.
Almost correct. IIRC, your Pup of choice is a highly modified version of one of, I think, the 4 Series; Lupu being among the 5 Series. I don't have a working Wary, Racy or Saluki to test. The only "script" I'm certain of is jpep's modification of the boot parameter. However, shinobar has been very active in modifying PupSaveConfig, http://www.murga-linux.com/puppy/viewto ... 081#457081. His recent modification, standard in Carolina and Carolina-Vanguard, includes the choice to Never Save with even the dialog at Shutdown being absent.
For hard drives and other fast media there is no ram for changes...the save file is used exclusively so any changes are instantly added.
That's the default setting you get on a frugal install when Grub4dos writes the Menu.lst. jpep's parameter modification and the user's configuration of SaveSession can change that.

The following has been my experience with Lupu and several of this year's litter of pups, frugally installed to hard drives, boot parameter changed to pmedia-ataflash, and SaveSession set to 0:
Change Wallpaper, don't save, old wallpaper appears on reboot.
Load SFS-on-the-fly, don't save, SFS isn't loaded on reboot.
Install pet, restart X if necessary, run pet, don't save, pet not present on reboot.

I admit being rather confused regarding how Puppies handle SFSes. From experience when an SFS is loaded, but not as yet opened, some portion of it exists in RAM. Usually there's a listing on the Menu. If, in fact, it has been loaded, even if there is no listing on the Menu, typing the name of its executable or wrapper in a terminal will either open the application or produce an error message other than "command not found."

Some time ago, in attempting to compare resource usage of installed pets, SFSes and Program Folders --applications decompress outside the SaveFile linked to the OS via scripts-- I used the then current LibreOffice as the test subject. As an SFS (with, I think gz compression) it required about 175 Mb of hard drive. As a decompressed Program Folder it took, IIRC, about 500 Mbs of hard drive. As an installed pet, it required the SaveFile use 500 Mbs of hard drive in addition to anything else in the SaveFile. As a loaded, but not opened, SFS it required about 35 Mb of RAM.

The "Urban Dictionary" gives other definitions, but the one I first learned for "blivit" was "ten pounds of shit in a 5 pound bag." :)

So here's my problem with how Puppies handle SFSes. I don't have a Swap File. And my computer has 4 Gbs of RAM --3.5 recognized running 32-bit OSes. So I can't explore the problem. What happens when you only have, say, 512 Mbs of RAM, your OS is using, say, 100 Mbs of that, and you attempt to open an application which requires 500 Mb? System crash? Application doesn't open? Something else?

The more important question is this: When an SFS is loaded and opened, the files you can observe if you had decompressed the SFS appear when you browse to their location. Or to be clearer, if you decompressed an SFS you can browse thru its files and folders and observe and delete or modify say its executable at /usr/local/bin. if, instead, you load the SFS and browse to /usr/local/bin you'll also see that SFS's executable. But what you are seeing is then only be in RAM. What happens if you delete that executable and --being in PUPMODE 13 configured not to automatically save-- shutdown without saving? My guess would be that the "deletion" would not have been written. That on reboot, the SFS will be decompressed from its "pristine" compressed file; still have an executable; and still be functional. But that's a guess.

"I'll be back" after testing.

mikesLr
Last edited by mikeslr on Mon 09 Feb 2015, 23:09, edited 1 time in total.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#130 Post by rufwoof »

I've messed around with 'loading' using sym links i.e. mount the sfs somewhere and then

cd /
cp -rs /mnt/somewhere/* .

.. and you end up with thousands of sym links under / and sub folders/files into the sfs

Really quick way to get a sfs up and available asap.

I've since found a way to mount sfs more directly into aufs and now use that. i.e. more correctly layers in the sfs - but doing so in the blink of a eye (thanks to a large part due to MikeB's guidance/pointers).

Not sure whether this is strictly correct/ok, but it works ok for me :

Code: Select all

#!/bin/bash
for i in sfs1.sfs sfs2.sfs sfs3.sfs; do
  f=`losetup -f`
  if [ ! -d /initrd/pup_$i ]; then
    mkdir -p /initrd/pup_$i
    losetup $f /initrd/pup_ro2/OFFICE/$i
    mount -r -t squashfs -o noatime $f /initrd/pup_$i
    busybox mount -t aufs -o remount,append:/initrd/pup_$i=ro unionfs /
  fi
done
fixmenus
Which loads in around 1.5GB of non compressed, 400MB of compressed (SFS's total file size) of extra apps/stuff in around a second or two (I have a fast version of fixmenus that someone kindly pointed me to but can't recall the exact details off-hand).

Note that code is specific to PUPMODE 5 i.e. I boot pupmode 5 all of the time and have disabled the shutdown save menu (run with no save file ever). In that mode pup layer is always set to be /pup_ro2 - and I have all of the sfs's that I load stored in /OFFICE on the ramdisk (so accessed in the above code as /initrd/pup_ro2/OFFICE/<sfs>

User avatar
RetroTechGuy
Posts: 2947
Joined: Tue 15 Dec 2009, 17:20
Location: USA

#131 Post by RetroTechGuy »

mikeb wrote:
How to decrease savefile size plz ?
you can use the resize2fs command on the unmounted save file specifying the new size...make sure its still big enough for what it contains. see the hlep/man page.

Mike
Has this always been in the Puppy versions? I didn't know it was there... Whoa! :D
[url=http://murga-linux.com/puppy/viewtopic.php?t=58615]Add swapfile[/url]
[url=http://wellminded.net63.net/]WellMinded Search[/url]
[url=http://puppylinux.us/psearch.html]PuppyLinux.US Search[/url]

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

The Invulnerability? of an SFS on a Harddrive

#132 Post by mikeslr »

Yesterday, having posed a question at the end of this post, http://murga-linux.com/puppy/viewtopic. ... 188#827188, I spent an unreasonable amount of time testing what effect, if any, operating without an automatic save making changes to the “copy of an SFS loaded into RAM

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#133 Post by rufwoof »

[quote]I don't believe data consigned to the “Swapfile

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

Re: The Invulnerability? of an SFS on a Harddrive

#134 Post by rufwoof »

[quote="mikeslr"]Maxwell, one of the greatest mathematicians of all time, argued that light was a wave traveling through aether. Subsequently, Einstein's mathematics “proved

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#135 Post by rufwoof »

Open a rox window and navigate to /initrd/pup_rw

Open another rox in your home directory.

Grab the Web-Server directory out of /root and drag that to outside of puppy, such as to /mnt/home so that its in effect 'deleted' out of /root. Look in the /initrd/pup_rw rox window and click the eye icon so you see hidden files and there's a whiteout file .wh.Web-Server that indicates that the Web-Server folder was 'deleted'. Move (drag/drop) the Web-Server folder back in again into /root and the /initrd/pup_rw whiteout file disappears (to indicate that the folder hasn't been deleted).

You can play around in a similar manner with other tests, such as create a file in /root and see how it also appears in /initrd/pup_rw. For new/added files a copy is also created in /initrd/pup_rw and if later that new file is deleted its also just deleted in /initrd/pup_rw (no need for whiteout files).

Mikeb guided me towards how you can create a savefile using the pup_rw layer. Copy that at shutdown, reinstate the copy at bootup and you have persistence of changes. Not all of the content however needs to be preserved, i.e. ignore /tmp ... and others. If you make a copy of /initrd/pup_rw using mksquashfs with no compression it copies (and restores) really quickly even if many changes have been made. Periodic remastering can enable the changes to be more permanently installed and the 'savefile' (sfs made using mksquashfs of the /initrd/pup_rw folder) reset to a zero size again.

I did use that approach for a while, but then changed things less and less as I got closer to the puppy I preferred so I resorted to not bothering and just remastering as/when I wanted to install a permanent change i.e. I just ram boot pupmode 5 each/every time and don't save (save option eliminated) at shutdown. I store all data outside of puppy space, use either a freshly downloaded new version of firefox, or a portable firefox on HDD, and use a online email account - so the core puppy remains the exact same from one reboot to another (read only so any virus can only persist for a single session). The main benefit for me is that I can play around, potentially totally screw up the puppy session/files - and then just reboot to be back at a pristine version of puppy again.

Jasper

#136 Post by Jasper »

Wife to husband: "And what are you going to do today?"

Husband: "Nothing."

Wife: "But that's what you did yesterday."

Husband: "I know, but I didn't finish it."

Alternatively:
some so-called professionals are frequently regarded a waste of space and good for nothing.

Finally, G H Hardy, a truly great English mathematician, observed in his famous essay "most people can do nothing well" and that statement was not qualified by "we expect, but cannot prove".
--------------------
My save file is 73B and Hardy may have written "men" not "people".

User avatar
mikeslr
Posts: 3890
Joined: Mon 16 Jun 2008, 21:20
Location: 500 seconds from Sol

Thanks rufwoof & Jasper

#137 Post by mikeslr »

Thanks, Jasper, for joining in the spirit of my "aside". During my youth, as my mind was developing, my slight astigmatism went unnoticed. It was only later that I put two and two* together and reached the conclusion that my view of the world, seen from the prospective of others, was somewhat askew.
Perhaps I am not alone.

Thanks, rufwoof, for the detailed explanations, and especially those regarding your method of utilizing Mikeb's techniques. Near the top of my "To Do" list is to carefully re-read ALL of your posts regarding how you've structured your Pups, and attempt to duplicate them: internalize the process rather than just store it away as interesting.

mikesLr

* There was a time when I had absolute faith in those parts of conclusions based on mathematical analysis. Math had been my strongest subject. When I was the eleventh grade a classmate, using algebra, demonstrate that Two plus Two equaled Two. It was only when I went thru each step he had taken, substituting real numbers for the algebraic symbols, that I realized concealed within his procedure was division by 0. After that I operated on the principle that if I could be fooled by a simple "slight of mind" trick, it could happen to the best of us.

User avatar
8Geee
Posts: 2181
Joined: Mon 12 May 2008, 11:29
Location: N.E. USA

#138 Post by 8Geee »

Pupmode 13 is OK for most general purposes. The ability to configure the Puppy Event manager and inactivity shutdown make it near ideal. A journaled filesystem like ext3 for both format and SFS can occasionally shrink by itself. I've seen Slackos do this on a smallish scale (my savefile was 96MB free, increasing to 99MB free out of 124MB availible). I believe this is due variable compression techniques (squashFS?).

The situation presented here CAN get to the point of snow removal in Boston. At some point one has to ship it out, much akin to garbage.
Linux user #498913 "Some people need to reimagine their thinking."
"Zuckerberg: a large city inhabited by mentally challenged people."

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

Re: Thanks rufwoof & Jasper

#139 Post by rufwoof »

mikeslr wrote:Thanks, rufwoof ...
Your welcome.

I have a Google drive that gives you 15GB of free space that can be used as a web server (see http://murga-linux.com/puppy/viewtopic. ... 170#794170). Mine is very spartan at present, just with my vmlinuz and initrd stored/presented there http://tinyurl.com/qhfdjfd

If you have grub4dos or similar installed then you can drop those in as another entry in menu.lst

i.e. I store my vmlinuz and initrd files for various pup's in different sub directories on /mnt/sda3 which is the first drive (sda) and third partition - which when counted from zero as the first, one as the second ... etc makes my root being hd0,2 and accordingly my menu.lst looks something like

title Rufwoofs
root (hd0,2)
kernel /RW/vmlinuz
initrd /RW/initrd.gz

i.e vmlinuz and initrd stored in /mnt/sda3/RW sub-directory.

Provided you have something like

timeout 5

near the top of menu.lst then you'll have 5 seconds when booting the PC to arrow up or down to whichever pup you want to boot

The initrd is quite large at around 430MB as it includes the full Libre Office (writer, calc, presenter, draw, database etc) with UK and US dictionaries, Openshot video editor with the associated Blender (3D animation) and full inkscape (titles editing), xvidcap for screen capture and Audacity for sound editing. Skype is also in there. All remastered to my PC but I find that equally boots on two other household PC's as-is (just select Vesa and 1024x768 resolution options during booting).

The normal puppy menu is accessed via right clicking the desktop - I have a smaller/simple menu as the normal bottom left menu click.

Based on Slacko 5.3.3, but with a later PAE kernel (3.10) - so supports larger ram systems and later PC's

No save option, just ram boots the same each and every time. If I want to make a change I usually boot a fresh/clean version, immediately make the desired changes and then use the menu, config, remaster option - which after running creates a new initrd in /root (which I then move to /mnt/sda/RW so its used at the next reboot). A copy of vmlinuz is also produced in /root as part of the remaster process, but that's the same as the original one and can just be deleted. Both vmlinuz and initrd are created under /root after a remaster so that PXE server can be fired up (menu, PXE Server) and other PC's on the LAN PXE boot using those.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#140 Post by rufwoof »

8Geee wrote:Pupmode 13 is OK for most general purposes. The ability to configure the Puppy Event manager and inactivity shutdown make it near ideal.
I originally found the pupmodes to be somewhat confusing/opaque and ended up using and sticking with ram boot (pupmode 5) and do a automated rotation into pupmode 12 as part of shutdown (and have no prompts for creating a savefile). I've also added some code to /etc/rc.d/rc.shutdown to spin down the drives as part of shutdown (based around hdparm -Y). Mine being rather old are a little noisy and you can quite clearly hear them spin down before a few seconds later the PC powering off. Parking the heads helps reduce the risk of corruption. Often I don't have drives mounted anyway - or manually umount them before I shutdown when they are mounted, so it wouldn't really matter if the power was just cut sharply.

Post Reply