Phoenix CE - Puppy full pup_save.2fs handler

[ecomoney]

Something that causes the most problems in puppy to the average user is when the pup_save.2fs file becomes full up. Using icewm this causes the desktop icons to disappear or xwindows refusing to start (Ive still not found a way of recovering them other than to do a reinstall).

Its definitely going to occur at some stage of the life of the installation. At the moment this isnt handled at all, and relies on the user reading the manual before trying puppy and running the resize utility to ensure that it doesnt happen, and rechecking disk space regularly (anyone who has ever supported end-users know that this just isnt going to happen!!!)

Im hoping someone who is knowledgeable with scripting would create a handler for this problem for 3.03ce. (It has already been reported here with some possible solutions).

Thanks in advance for your help.

[ecomoney]

Ive been doing some reading on the developer blog, and found out about script called /etc/sbin/savepuppyd that runs constantly in memory and checks for when the pup_save file needs to be written to (i.e. when *memory* is full). Could it be modified to keep track of available save file space?

Code: Select all

#!/bin/sh
#(c) Copyright Barry Kauler 2007 www.puppylinux.com
#2007 Lesser GPL licence v2 (http://www.fsf.org/licensing/licenses/lgpl.html)

#saves current session, and monitors memory space.
#run as a daemon, so will save the tmpfs periodically, also if tmpfs getting full.
#launched from /etc/rc.d/rc.local0, at low priority 19.

#v2.21 13 Sept 2007  BK: updated.

#variables created at bootup by /initrd/usr/sbin/init...
. /etc/rc.d/PUPSTATE

PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R7/bin"

#only run if need to save tmpfs periodically...
SAVETMPFS="no"
[ $PUPMODE -eq 3 -o $PUPMODE -eq 7 -o $PUPMODE -eq 13 ] && SAVETMPFS="yes"
[ "$SAVETMPFS" = "no" ] && exit
#note, we don't have auto-save to multisession-cd (PUPMODE=77) -- have a desktop icon
# that runs savesession-dvd, also at rc.shutdown.

sleep 300 #5 minutes. 600 delay for 10 minutes.

LOOPDELAY=6
SLEEPTIME=300 #save every 30 minutes.

CNTSPD1=1
while [ 1 ];do
 TMPFREEK=`df -k | grep '/initrd/pup_rw' | grep 'tmpfs' | tr -s ' ' | cut -f 4 -d ' '`
 [ ! $TMPFREEK ] && exit 1 #precaution
 #i want a desktop save button to be able to trigger a save, via /tmp/snapmergepuppyrequest...
# if [ $CNTSPD1 -ge $SLEEPTIME -o $TMPFREEK -lt 500 -o -f /tmp/snapmergepuppyrequest ];then #every 30 or 5 minutes, or <500K or by request.
 if [ \( $CNTSPD1 -ge $SLEEPTIME -a ! -f /root/Choices/NO_30min_SAVING \) -o $TMPFREEK -lt 500 -o -f /tmp/snapmergepuppyrequest ];then #every 30 minutes, or <500K or by request.
  [ -f /tmp/snapmergepuppyrequest ] && rm -f /tmp/snapmergepuppyrequest
  RETVAL=1
  CNTSPD1=1
  RUNPS="`ps`"
  XPATTERN='gxine|petget|wget|axel|dotpup'
  if [ "`echo "$RUNPS" | grep -E '$XPATTERN'`" = "" ];then
   yaf-splash -display :0 -font "8x16" -outline 0 -margin 4 -bg orange -placement top -text "Saving RAM to 'pup_save' file..." &
   RETVAL=$?
   YAFPID=$!
   sleep 3
   sync
   /usr/sbin/snapmergepuppy >> /tmp/SMP-output.txt #/initrd/pup_ro1 /initrd/pup_rw
   [ $RETVAL -eq 0 ] && kill $YAFPID
  fi
 fi
 CNTSPD1=`expr $CNTSPD1 + 1`
 sleep $LOOPDELAY
 [ $TMPFREEK -lt 500 ] && sleep 300 #avoid 6-sec saves loop if out of memory.
done

xmessage "/usr/sbin/savepuppyd activated!!!"

###END###

you will notice Ive added a simple message at the end to see when its called, but it hasnt on my frugal install as yet.

I was contacted again by someone whos computer I support last night with the disappearing icons bug (and it is a bug!!!), it also recently happened to my new company secretary. Im getting about one of these callouts a week.

[cb88]

it only runs when you use puppy on a flash drive and it monitors the ramdisk I think doesn't it? I see where you are going though...

[Pizzasgood]

Code: Select all

#only run if need to save tmpfs periodically...
SAVETMPFS="no"
[ $PUPMODE -eq 3 -o $PUPMODE -eq 7 -o $PUPMODE -eq 13 ] && SAVETMPFS="yes"
[ "$SAVETMPFS" = "no" ] && exit 

That chunk kills it if you aren't in PUPMODE 3, 7, or 13. 13 is booting with a pup_save.2fs file on flash media. I don't remember the others, but they're on the "How Puppy Works" page of Barry's site, and probably a script or two within Puppy.

You could easily modify that to keep running, but only actually do the flushing for the correct modes. Then just add the code you want in the part that does get run each cycle.

Otherwise, you could just make your own daemon that runs all the time. It would be another process to show up in ps, but wouldn't hurt performance any (it would spend most of its time sleeping).

[ecomoney]

You could easily modify that to keep running

Im afraid Im not easily able to do many things in puppy, apart from find out what needs doing!

How often does this script run? every 5 minutes? Would there bi a big performance overhead to it running say...every thirty seconds (most users, when copying things to their pup_save.2fs file would wait at least this long if there was no activity for that amount of time).

another way I have thought of doing this is, before x is started, write a small empty file, and then check if it exists. If not, then there isnt enough space to start. I think then theres a command under bash to list the most recently accessed files, and give the user an opportunity to delete them before rox is started. Also a file could be written on shutdown, that could be delete on the next bootup to ensure the xorgwizard and desktop icon scripts were given the opportunity to write. If the file couldnt be written at shutdown that means that the pup_save file is full and an option could be given to make the pup_save file bigger or delete recent files (I believe a small file with the number of bytes to increase the filesize by is written to root to make this happen?). Personally Ive never had a problem with large pup_save.2fs files (even over 2gb),

If anyone would be willing to contribute some "quick and dirty" code I would be happy to take the time to test it out. Whatever the solution, anything would be better than nothing at the moment! Puppy is currently an accident waiting to happen.

[Pizzasgood]

It would be best not to mess with the frequency of that. Better to just make another daemon.

Here's an example template, which you'd start with an ampersand to background, and would probably set to a very low priority (don't remember how off the top of my head, and not inclined to look it up at 3AM. It involves nice, or maybe renice.)

Code: Select all

#!/bin/sh

#This should either be started with low priority, or modified so that it sets
#itself to run at low priority.  That way it doesn't interefere with anythign.

#this is how much delay between cycles (in seconds)
DELAY=30

#this is the loop
while [ 1 ]; do
	
	
	
	## Put code to be looped in here ##
	
	
	
	#take some time out before starting again
	sleep $DELAY
done

And it looks like this code shows how much free space is left in the save file (or whatever happens to be mounted on /initrd/pup_rw):

Code: Select all

df -k | grep '/initrd/pup_rw'| tr -s ' ' | cut -f 4 -d ' '

[ecomoney]

Hi Pizzasgood

I typed that line into the console and got the sizes of the pup_save.2fs file remaining and used. Should it be a matter of doing some string handling to extract the two figures from that line and do a comparision, that way the program can either perform the resize (or whatever other action) or not.

I know nothing about bash scripting, but a bit about efficient programming. this script needs to be run frequently so it should be kept as small as possible. It only needed in a frugal install, so perhaps the script that runs at the beginning (say /etc/rc.d/rc.local?) would check the pupmode, and if its set to a frugal install then start the script, his would reduce overhead in other types of install.

As I say Im an old vb programmer, so the code in vb would look like

Code: Select all

sizedata$ = df -k | grep '/initrd/pup_rw'| tr -s ' ' | cut -f 4 -d ' '
megremaining = cint(trim(instr(sizedata$, 30, 10)))

if megremaining < 5 then
     #Display warning message and suggest solutions here#
endif

Can anyone translate?

[Pizzasgood]

That's odd; it works on my system. All it is is a slight modification of one of the lines in the original code you posted. Did you copy and paste it, or type it out yourself?

Could be an issue with newer/older Puppies, as I only tested it on 2.14 (Pizzapup).

Also, there is a difference between Live CD with a pup_save.2fs and a Frugal install. Frugal is actually putting Puppy on the harddrive, with Grub. Things are already confusing enough between Normal/Full and Frugal, so calling LiveCD 'Frugal' will only worsen matters.

Aside from LiveCD and Frugal, USB also uses a pup_save.2fs file. So it would have to run for any of those three. That means PUPMODE 12 and 13. Techinically, it could be 9,10,11,12, and 13, but as far as I know only 12 and 13 are going to be seen in normal situations. It wouldn't hurt to check them all, since it would only be done once per boot. Something like this:

Code: Select all

. /etc/rc.d/PUPSTATE
if [ $PUPMODE -ge 9 ] && [ $PUPMODE -le 13 ]; then

#   ONLY DO STUFF IN HERE
#   IF CORRECT VERSION

fi

This is the whole thing, other than that it's still using the same size finding thing I posted.

Code: Select all

#!/bin/sh

#This should either be started with low priority, or modified so that it sets
#itself to run at low priority.  That way it doesn't interefere with anythign.

#this is how much delay between cycles (in seconds)
DELAY=30 #seconds

#this is at which amount of free space (in kb) to nag the user:
NAG_SPACE=5000 #kb

#this is how long between nags (give the user time to do something, so it
#doesn't nag them continuously.  Set to -1 to only nag once.
NAG_INTERVAL=-1 #seconds

#load this file to get the PUPSTATE variable, which tells us
#whether there's a pup_save.2fs file or not
. /etc/rc.d/PUPSTATE
if [ $PUPMODE -ge 9 ] && [ $PUPMODE -le 13 ]; then

	#this variable is so we don't nag over and over before the
	#user gets a chance to do anything about it
	LAST_NAG=0
	#this is the loop
	while [ 1 ]; do

		#the free space left in the save_file
		SIZE_REMAINING=`df -k | grep '/initrd/pup_rw'| tr -s ' ' | cut -f 4 -d ' '`
		
		if [ $SIZE_REMAINING -le $NAG_SPACE ] && [ "$NAG_INTERVAL" != "STOP" ]; then
			NOW=`date +%s`
			LAPSED=`expr $NOW - $LAST_NAG`
			if [ $LAPSED -ge $NAG_INTERVAL ]; then
				[ $NAG_INTERVAL -lt 0 ] && NAG_INTERVAL="STOP"
				LAST_NAG=$NOW
				
				##### WARNING MESSAGE #####
				DURATION=20 #seconds
				yaf-splash -timeout $DURATION -fg black -bg orange -margin 20 -font "9x15B" -outline 0 -text "WARNING

Your pup_save file only has $SIZE_REMAINING kb of free space left.
You should refrain from putting anything else in it until
you either delete some stuff or resize it from the menu
( Start -> Utility -> resize personal storage file )

This message will self destruct in $DURATION seconds
or whenever you click it, whichever comes first."
				
			fi
		fi
	   

		#take some time out before starting again
		sleep $DELAY
	done

fi

The important variables are DELAY, NAG_SPACE, and NAG_INTERVAL. The DELAY is just how long between loops. The NAG_SPACE is for how much free space to have remaining before it complains (in kilobytes, same as df). The NAG_INTERVAL is how long between successive nags (so if you set it to 300, it will only nag you every five minutes). If you set it to -1, it will only nag once per boot.

Note: All times are in seconds, and all sizes in kb.

[ecomoney]

im at a (happily puppied) friends house now, but Im looking forward to testing this out! ive had two callouts in the last week with this problem - which is a pain when my car is off the road!

[littlelollypoplady]

as a lowly end-user and someone who is not overly experienced with using pcs i found the entire experience of the puppy experiment awful from start to finish.....if there has to be a problem with storage space that will render your pc unusable until someone who knows how to reinstall has been to fix it, why not design a handy wizard where something can pop up to warn you that you are running out of disk space and a tutorial that guides you through re-sizing your storage space so that we inexperienced end-users can run puppy without getting someone to hold our hands through it...or is that the idea that only experts can use puppy effectively........rob has now installed linux mint and it seems much better than puppy.....so far.....i just want to be able to use the internet...i do not care a great deal either way who runs my pc between linnux or microsoft...so long as it does the job without causing me too much stress...i like the idea of open sourse....so long as it actually works!!!!!

[tronkel]

Dear Littlelollipoplady

If all you're doing is internet surfung, you should be able to do that by booting from live CD with the kernel cheat code "puppy pfix=ram" (no quotes) - ie type that in at the boot prompt on the first screen that you see when the CD has loaded. This way you are not involved with any pup_save file that can fill up.

If you need to save any files, you can do that direct to a USB stick or your normal hard disk. Make sure you use PMOUNT to mount the drive first though.

Any pup_save file can get used up if you install a lot of extra software, but if all you do is surf the net, you probably won't be doing that anyway. Doesn't get much easier than that with any other Linux or Microsoft system.

Please take care of any Scunthorpe puppies that may use your services to cross the road safely! Bow Wow!

[2byte]

The lady makes a valid point. Let he who has ears listen.

Extremely friendly for Linux newbies.
Will just work, no hassles.
These are extraordinary goals, yet Puppy achieves them all.

[inflammatory remarks]
Nope. Sorry. Linux newbies take a back seat with Puppy. No hassles? A distro that becomes unusable if two movies are saved or trashes your existing Puppy setup if the user is unaware that a new version he/she tries will 'upgrade' it without warning cannot be called extremely friendly for Linux newbies. Disregarding issues like these in a distro as old as Puppy is inexcusable. Especially when an issue can be fixed with a dozen lines of code.
[/inflammatory remarks]

[ecomoney]

Thank you Becky for taking the time to tell us all about this MAJOR problem (especially seeing that you have your arm in plaster at the moment!). This wasnt pinpointed at the cybercafe as people there dont generally save files, but in a home environment its a different story entirely. 2Byte I think you sum up the problems with puppy very well (the upgrade problem is also pretty severe). I cant think of many of my users that havnt encountered it at some point.

I think the reason this problem isnt reported more is, when a puppy user encounters it, it is "ENDGAME", the computer isnt useable to make the report, and the person it effects is probably not feeling inclined to come back and report it (even if they identify the problem) after having to rebuild their computer setup (probably with windoze ).

From the severity and frequency of this problem to the people that puppy is aimed at, I dont think its right to release a community edition until this problem is fixed satisfactorily , as it will only serve to give people a negative experience of open source. Its easilly puppy enemy number 1!!!

[2byte]

ecomoney,

I have been working on a new version of freememapplet that may help solve the pup_save issue. It differs from v1.2 in these aspects:
It is a true tray icon application. It will work in icewm and other window managers that support eggtrayicon. That includes jwm. v1.2 would not.
It displays an icon with colored backgrounds instead of text.
The available file storage space is read in the tool-tip.
The background colors are,
green: space >= 20MB
yellow: space < 20MB and > 5MB. Tool tip adds the word 'low'.
flashing reds: space < 5MB. Tool tip adds the word 'critical'.
These values can be changed in the code if need be.
There is also a new help file to go with it.
Also the 100% CPU usage of v1.2 is fixed.

As it stands now it does little to actively prevent a hung system startup but it would be a minor bit of coding to create a buffer file of xMB whenever the storage space reaches 25MB or so. The init script could detect and delete that file to insure enough room to boot, and I don't think anyone could save something fast enough to prevent its creation. It could also pop up that nag screen. The help file could also be expanded to include specific instructions to handle the situation.

This one has been tested in Puppy 2.14, 2.17, and 3.01. All with icewm and jwm.

Size is always an issue, this version adds 6.2kb over the existing one, plus a few kb for the help file.

Thoughts? Is this worth pursuing?

2byte

P.S. I also have a bullet proof (?) init script mod for 3.01 that prevents accidental version upgrades. I'll start a new thread for that.

[cb88]

I have already mentioned this before but seems apropriate to remind everyone.

If you create a save file that is too small for what you have installed with (puppy pfix=ram) puppy will not boot.... because the pupsave will be full and not all files will be availabe because they couldn't be copied before shutdown...

[Leachim]

I plan to write a completely new save-system that will address most of the problems mentioned in this thread.

Most of all I want to address the problem that saving a lot of changes to a pen-drive can nearly hold the system for several minutes - each 30 minutes.

The new system shall allow stepping back to some earlier point in time, so if the latest changes rendered your system unusable, you can still continue with some earlier recovery point.

I want to start the development of the new save-system somewhere in February. (I hope to finish some other projects till then.)

[willhunt]

it would be nice if puppy had a e2fsck during boot
or a pfix=e2fsck option so it ran it at startup
then new user would not have messed up
pup_save files that lead to lottsa problems
later and it's goota be better than the pfix=ram
thing and it would save a boot cycle or two

[Dougal]

I have been working on a new version of freememapplet that may help solve the pup_save issue. It differs from v1.2 in these aspects:
It is a true tray icon application. It will work in icewm and other window managers that support eggtrayicon.

Any chance you can give the eggtrayicon treatment to AbsVolume, too?

[HairyWill]

If you just put Absvolume into eggtray the volume slider will not resync to the current volume when you restore the window.
Absvolume only sets the slider postion by reading the volume when it is started. This isn't a problem if you only ever use absvolume to control the volume.
The way that I hacked absvolume for dingo has a few weaknesses:
1. Dingo alpha 2 doesn't actually have the latest version in.
2. Icon transparency doesn't seem to work in dingo though it does in 3.01 as long as you restart X I will probably externalise the images to address this problem.
3. When the volume control is open the window is visible on the taskbar. You can create a jwm rule to disable this, I presume this is possible in icewm as well.

Things I have changed about it that I like are:
1. an icon that changes with the volume
2. the tooltip also contains the volume

Criticism will be warmly received.
http://users.ecs.soton.ac.uk/wmd04r/pup ... py-0.4.tgz
I thing there is a precompiled binary in the tar as well as the source

[ecomoney]

Hi Leachim/All

Thanks for putting your talents to work on this one as well. Ive been doing some research into this fault and have found a few things out. It appears that the fault is caused by a file called "PuppyPin" that is the configuration file for the ROX pinboard (i.e. desktop) It gets copied from one place to another on startup (in rc.local?) except when there isnt space to do that. If this happens then the file isnt moved to where it usually apepars (in /tmp ?) and the icons disappear from the desktop.

Sorry for being vague, but this is my understanding as much as it is.