How to modify rc.shutdown for automatic backups?

[PaulBx1]

I'm trying to modify rc.shutdown to do an automatic backup of my pupsave. However I cannot figure that script out enough to do it. I'm wondering if someone here is familiar enough with that script to give a hand.

I need a point in the script where the following is true:
1) The pupsave is no longer mounted
2) Everything else still is

I'm not even sure if that is possible. I can't figure out where the pupsave is unmounted.

My idea is to add a variable to PUPSTATE, something like PUP_SAVE_BACKUP_DIR='initrd/mnt/dev_save/backups/puppy'

rc.shutdown would then (after umounting the pupsave) check for the existence of this variable. If it existed and was non-null, it would cp the pupsave to that location. It would also be able to keep two or three older versions of the same thing down there simply by doing a rename, e.g.
"mv $PUP_SAVE_BACKUP_DIR/$SAVEFILE_2 $PUP_SAVE_BACKUP_DIR/$SAVEFILE_3"

The whole point of this is to avoid having to manually boot pfix=ram and backing up the pupsave (which means it rarely gets done). I lobbied for this in Puppy 4.2 but it didn't get in.

[WhoDo]

I lobbied for this in Puppy 4.2 but it didn't get in.

It's certainly a great idea. If it had been built in time for Puppy-4.2, it would have gone in, Paul. One of my regrets is that I don't have the time or the inclination to get back into programming again and I don't know Bash, or I would happily help with this project. If you can get it done on your own, or with some help from one of the devs, I'll be happy to help test it for you and we can take it from there.

[trio]

Well, good idea..

but what you're talking will have something to do with altering initrd.gz, not only rc.shutdown...maybe something more...

[Bruce B]

I'm trying to modify rc.shutdown to do an automatic backup of my pupsave. However I cannot figure that script out enough to do it. I'm wondering if someone here is familiar enough with that script to give a hand.

I need a point in the script where the following is true:
1) The pupsave is no longer mounted
2) Everything else still is

{ rest cut }

PaulBx,

Wonderful idea. If we making backing up easy work, more
people will routinely back up.

An independent script or a modified rc.shutdown would no
longer be available if pup_save were unmounted.

Good backups can be made of an active pup_save file, but
because it is subject to change, I don't know how to
assure a bit for bit perfect backup under conditions where
changes may occur.

An inactive pup_save file, we can reasonably assure a bit for
bit perfect backup.

What do you think of making a second pup_save file, say
about 16MB in size which contains the scripts for
automatically performing maintenance, backups and
rebooting after the work is done?

This way the user determines the intervals merely by
booting that pup_save file and it does the rest.

? ? ?


Bruce

[Bruce B]

rc file first draft


Q: intuitive enough

Q: comments on options

Q: like it - hate it

Code: Select all

## ~ rc file example first submission 
## ~ this submission has an intentional limitation, it only 
## ~ backups one pup_save file


## ~ to active options fill in correct data and remove the #

## ~ name of pup_save file (required)

# savefile pup_save.xyz

## ~ pup_save directory  (required) (no trailing \'s)

# pupdir /initrd/mnt/dev_save/abcdefg

## ~ save files in this directory, not required, default is pupdir above

# backupdir /foo/bar

## ~ number of backups to maintain, not required default is 2

# numbackups 1

## ~ e2fsck prior to backups, not required, default is yes

# filecheck no

## ~ make compressed backups, not required, default is no, if yes is 
## ~ choosen, the full e2fsck is implicit, defaulting to yes
## ~ note this option if yes is time consuming and space saving

# compression yes

~

[rcrsn51]

@PaulBx1: I have also looked at the rc.shutdown script, hunting for a point where I could work with the pup_save file. But I could never find it, and I suspect that the problem is related to the fact that the pup_save doesn't appear to get unmounted cleanly. The evidence for this is seen by how e2fsck will routinely identify errors in the pup_save's filesystem.

However, I have found a point in the init script where you can manipulate the pup_save during boot-up. Read here.

You could modify the technique to backup the pup_save at the start of the next boot instead of at the end of the session.

[PaulBx1]

I'm glad I'm not the only one having trouble identifying where pupsaves are unmounted in rc.shutdown. I was beginning to wonder about myself. I am not really a bash programmer either, but sometimes I can cobble something together.

An independent script or a modified rc.shutdown would no
longer be available if pup_save were unmounted.

Yes. But once you are executing rc.shutdown, isn't it already in memory? So unmounting the pupsave does not matter? If that doesn't work, then some small script could be copied to ramdisk and executed there. Or even the bulk of rc.shutdown could be executed that way (what I mean is, rc.shutdown becomes a script that does nothing but copy the subsequent script to ramdisk and transfers control to it).

Good backups can be made of an active pup_save file

I have used this with my convert-pupsave script, but I always felt a little iffy about it, and note that Barry went to the effort of using an inactive pupsave in his pupsave resizing utility.

What do you think of making a second pup_save file, say
about 16MB in size which contains the scripts for
automatically performing maintenance, backups and
rebooting after the work is done?

This way the user determines the intervals merely by
booting that pup_save file and it does the rest.

I want to make backups automatic, or at least backups that are simply enabled by selecting a different option in the shutdown menu. Making a user reboot is exactly what I want to avoid, because it means backups won't happen often. Backup is drudgery that computers should do, not humans.

I think doing it in shutdown is better than in bootup, because the user then can decide whether or not to back up, by which shutdown option he chooses; then he walks away from the computer and lets it do the work. If it were in startup, it would mean much longer boots, which again means people would tend not to do it as much as they should. The other thing is, you normally want to back up after you have done some work. Waiting for next boot makes it slightly more probable that work will be lost in the meantime.

BTW, we don't really need everything else to be still mounted; we can remount the backup target as we need to. But we still need to know where the pupsave is unmounted (if it is ).

I suppose we could do it at the very end...

[PaulBx1]

OK I found where it is unmounted. It is this code:

Code: Select all

 busybox mount -t $SAVEFS -o remount,ro $SAVEDEV /initrd${SAVE_LAYER} 2>/dev/null
 umount-FULL -i -n -l /initrd/${SAVE_LAYER} 2>/dev/null #-l is lazy unmount.
 

I surrounded those lines with "mount|grep loop > /dev/console" and found that the umount does indeed unmount the pupsave. (Interestingly, the remount command does NOT remount the pupsave read-only; it is still "rw" after that command.) BTW I put a "sleep 30" down at the end so I had time to inspect the results.

I thought I'd try "cp -a ${SAVEFILE} ${SAVEFILE}backup" after that umount to see if I ended up with a backup of my pupsave file, but it did nothing. Not sure why. This command works for me in an rxvt window. Hmmm, I wonder if my backup target volume was unmounted by that time? More investigation needed... It would be strange if the backup target was unmounted, since that is what the pupsave resides on!

<later>
Nope. /initrd/mnt/dev_save is still mounted. I changed that cp command to "cp -a /initrd/mnt${SAVEFILE} /initrd/mnt${SAVEFILE}backup", and put a "sync" after it too, but it still did nothing. Any ideas?

[Bruce B]

I'm glad I'm not the only one having trouble identifying where pupsaves are unmounted in rc.shutdown.

Although the command is there, I didn't think it did get unmounted cleanly.

Need to check on that

An independent script or a modified rc.shutdown would no longer be available if pup_save were unmounted.

Yes. But once you are executing rc.shutdown, isn't it already in memory?

Good catch, here is a proof of concept:

Code: Select all

#!/bin/bash
echo "Hello, $0 exists"
echo "now ready to delete $0"
rm $0
echo "If you can read this, $0"
echo "has been deleted, and I'm still working"
sleep 5

[Bruce B]

BTW - I wrote my own shutdown script and have modified
so many things. I wrote a script script to replace rc.shutdown.

I think you'll find rc.shutdown does not shutdown, rather prepares for the shutdown or reboot.

The calling program is probably /sbin/reboot or /sbin/poweroff or something similiar. I removed it.

The command for the actual reboot is /bin/busybox reboot

[PaulBx1]

Bruce, looks like you commented to my previous post; things have moved on in the meantime (our posts crossed each other, or however you want to say it). I'm now trying to do a cp in rc.shutdown and having trouble doing it. See above...

[Bruce B]

Paul,

Here is my pup_save file

/initrd/mnt/dev_save/dingo/pup_save.3fs

A simple copy command would be

cp /initrd/mnt/dev_save/dingo/pup_save.3fs \
/initrd/mnt/dev_save/dingo/pup_save.bak

If it were not a forum post, I would show it in one line.

Actually at 1024x768 it doesn't wrap on me so here is a
one liner:

cp /initrd/mnt/dev_save/dingo/pup_save.3fs /initrd/mnt/dev_save/dingo/pup_save.bak


Are we on the same page? Questions? Comments?

Bruce

[Bruce B]

Bruce, looks like you commented to my previous post; things have moved on in the meantime (our posts crossed each other, or however you want to say it). I'm now trying to do a cp in rc.shutdown and having trouble doing it. See above...

Sunday, morning, after an all nighter. I pleased I can even read. Sequence, what's that? I'm good to go, but the mind faded out somewhere between then and now. When ever 'then' was, 'cause I don't know.

[PaulBx1]

Looks like I'm as out of it as you were. I forgot the "/dev_save"!

Anyway, I have my backup file. So now the basic picture is there. Next, the user interface...

I was going to specify the location of the backup in PUPSTATE, which means the user edits that. Make sense? Isn't PUPSTATE generated in shutdown, at least the first shutdown? Is it replaced in an update? Maybe the location (and any other options like how many backup copies to keep) should be saved in an ordinary text file, like /etc/rc.d/pupsavebackupspecs.txt or something?

Then I was going to modify .jwmrc to have a separate "Menu>Shutdown>Poweroff with backup" option; also a "Reboot with backup". Make sense? Not sure how to invoke rc.shutdown differently, maybe passing a parameter or something.

[Bruce B]

My reboot script contains lots of on screen information.
This way I can see things I want to know.

Here is the final question for confirmation to reboot. I use
colored echo output. cecho would not be available in standard Puppy.

Code: Select all

	cecho "Reboot NOW (y,n)?" yellow
 	read -n 1
	[ "$REPLY" = "y" ] && /bin/busybox reboot 

The idea expressed here is: As it is going down you can
have any options you want, until the final OK to reboot,
then it's the real thing and you canceled your vote for
additional options if the choice is y

[Bruce B]

Looks like I'm as out of it as you were. I forgot the "/dev_save"! :roll:

Nothing to worry about, I'm confident with persistence and
patience and teamwork we will manage to put together a
few of functional lines of code.

Also the factor of helpful peer review and interest.

Five lines of code, five weeks should be adequate, I think,
but maybe I'm overly optimistic.

I'm wondering if Delaware would be a good place to
incorporate? We could hire a commity [sic], if we bankrupt
ourselves, we can live under Delaware's umbrella long
enough refine the script and market it.

[PaulBx1]

You may be overly optimistic.

The problem is, rc.shutdown is a mess in need of a thorough cleanup. Lots of barnacles have accumulated over time. If anyone takes on this task, I will have suggestions.

In the meantime, this is what I did. First, locate this line in rc.shutdown:

Code: Select all

 umount-FULL -i -n -l /initrd/${SAVE_LAYER} 2>/dev/null #-l is lazy unmount.

After that line, put this:

Code: Select all

 . /etc/rc.d/PUPBACKUP
 if [ "$PUPSAVEBACKUPDIR" ]; then
  echo -n "Back up your pupsave to ${PUPSAVEBACKUPDIR}? : " > /dev/console
  read
  if [ "$REPLY" = "y" ]; then
   mkdir -p $PUPSAVEBACKUPDIR
   while [ $PUPSAVEBACKUPCOUNT -gt 0 ]; do
    mv ${PUPSAVEBACKUPDIR}${SAVEFILE}$[ PUPSAVEBACKUPCOUNT - 1 ] ${PUPSAVEBACKUPDIR}${SAVEFILE}${PUPSAVEBACKUPCOUNT}
    let PUPSAVEBACKUPCOUNT=PUPSAVEBACKUPCOUNT-1
   done
   echo "${SMNTPT}${SAVEPATH}${SAVEFILE} being copied to ${PUPSAVEBACKUPDIR}${SAVEFILE}0" > /dev/console
#  cp -a ${SMNTPT}${SAVEPATH}${SAVEFILE} ${PUPSAVEBACKUPDIR}${SAVEFILE}0
   cp -a /initrd/mnt/dev_save${SAVEFILE} ${PUPSAVEBACKUPDIR}${SAVEFILE}0
   sync
  fi
 fi

(Watch out for those two lines that wrapped.)

Also in /etc/rc.d put a file called "PUPBACKUP" containing these lines:

Code: Select all

PUPSAVEBACKUPDIR='/initrd/mnt/dev_save/backup/pupsaves'
PUPSAVEBACKUPCOUNT=3

Then it should back up for you. This works for my setup (boot from CD, pupsave on the hard drive, Puppy 4.1.2). Try not to left click on rc.shutdown as it hangs your system.

It saves one more than you ask for, a minor annoyance. The source directory of the pupsave is hard-coded in, which is ugly, but SMNTPT and SAVEPATH are not defined normally; I'm guessing they are only there for the initial reboot.

Maybe someone can improve this. What with the mess rc.shutdown is in, my poor command of bash, I know it ain't pretty.