Reduce & speedup shutdown writes to USB with rsync

How to do things, solutions, recipes, tutorials
Post Reply
Message
Author
billstclair
Posts: 106
Joined: Mon 27 Feb 2006, 01:23
Location: Upstate New York
Contact:

Reduce & speedup shutdown writes to USB with rsync

#1 Post by billstclair »

I did this in Puppy 1.0.8. It will likely work, with slight script changes, in earlier versions. Puppy 2.0 works differently in this regard so probably doesn't need any such patch.

I'm running Puppy (1.0.8) from a 1 gig USB key. I let Puppy do the USB install for me. I noticed that shutdown took a long time "writing back to the pupxxx file on the Flash drive from ramdisk", making the USB access light flash the entire time. I found the responsible code in /etc/rc.d/rc.reboot:

Code: Select all

 rm -fr $PUPMNTPOINT/*
 #...heh, heh, this leaves behind hidden files/dirs, so do this...
 rm -fr $PUPMNTPOINT/.[a-zA-Z0-9]*
 sync
 cp -af /root/. $PUPMNTPOINT
 sync
This deletes all files from the pup100 file system on the USB key and copies the versions in the ramdisk back to there. If you have lots of extras loaded, this is a fairly large amount of data. My installation, for example, with Emacs, Firefox, and a few other smaller packages, is about 146 megs.

It's really only necessary to copy files that have changed. rsync is a standard Linux program that excels at doing exactly that. This HOWTO describes installing rsync and changing Puppy's startup and shutdown scripts to use it instead of cp to save the changes to /root.

rsync is not part of the standard Puppy installation. It isn't even available, yet, as a dotpup.

To build it, you need to get usr_devx.sfs, put it in your HOME directory (the same place as the pupxxx file), and reboot. Then download the rsync source from http://www.samba.org/rsync/. The latest stable version at this writing is rsync-2.6.7.tar.gz. Put that file somewhere, e.g. /root/my-documents. Then do the following:

Code: Select all

cd /root/my-documents
tar -xzf rsync*.gz
cd rsync*
./configure
make
make install
This puts an executable file, rsync, in /usr/local/bin.

Alternatively, you can download the attached rsync.gz, gunzip it, and mv it to /usr/local/bin. That should work for 1.0.8, where I compiled it, but may not work for other versions of Puppy.

Once you have rsync installed, you can back up /root with the following commands:

Code: Select all

rsync -a --delete /root/ /mnt/pupxxx
sync
The rsync command makes /mnt/pupxxx a copy of /root, copying as little as possible to make it so. sync flushes buffers to the storage medium. You can easily put this into a script file and bind it to a desktop icon to allow incremental saves whenever you want. Or use the crontab to schedule it at regular intervals.

To change Puppy's startup and shutdown scripts to use rsync for the copy:

Code: Select all

cd /root
cp /etc/rc.d/rc.reboot rc.reboot.rsync
Now edit rc.reboot.rsync, and change the code listed above to:

Code: Select all

# rm -fr $PUPMNTPOINT/*
 #...heh, heh, this leaves behind hidden files/dirs, so do this...
# rm -fr $PUPMNTPOINT/.[a-zA-Z0-9]*
 sync
# cp -af /root/. $PUPMNTPOINT
 /usr/local/bin/rsync -a --delete /root/ $PUPMNTPOINT
 sync
Finally, add the following to the end of /etc/rc.d/rc.local:

Code: Select all

# Use rsync to copy /root/ to /mnt/pupxxx
cp -f /root/rc.reboot.rsync /etc/rc.d/rc.reboot
cp -f /root/rc.reboot.rsync /tmp/rc.reboot
You can't just edit rc.reboot in place in /etc/rc.d because Puppy overwrites it with the virgin copy on every boot. The /tmp copy is the one that is actually used at shutdown or reboot time.

The first shutdown after making these changes will do the copy as usual. Subsequent shutdowns will be much quicker and will write much less to your USB key.
Last edited by billstclair on Tue 14 Mar 2006, 04:44, edited 1 time in total.

billstclair
Posts: 106
Joined: Mon 27 Feb 2006, 01:23
Location: Upstate New York
Contact:

#2 Post by billstclair »

One caveat. When you update your Puppy, you'll need to remove or comment out the changes to /etc/rc.d/rc.local before doing the update, then remake /root/rc.reboot.rsync by editing the updated version of /etc/rc.d/rc.reboot, and restore the rc.local lines. Or maybe 1.0.9 will break this...

I haven't checked how Barry does the incremental backups in Puppy 2.0. That may benefit from rsync as well...

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#3 Post by Flash »

Incremental backups are a good idea to save space and time, but it seems like it would also be a good idea to include an occasional "full backup", like MPEG saves a full frame every so often.

billstclair
Posts: 106
Joined: Mon 27 Feb 2006, 01:23
Location: Upstate New York
Contact:

#4 Post by billstclair »

I don't understand how the concept of "full backup" applies here. The RAM disk is copied from the pup100 file at startup. Then some incremental changes are made to the RAM disk. These need to be copied back to persistent storage. rsync ensures that the pup100 image exactly matches the RAM disk. Barry's erase and copy method DOES improve locality, which might matter if you were writing to a disk, but FLASH RAM has no locality issues that I know of.

Personally, I don't get a warm fuzzy feeling from knowing that my system exists only in RAM during the time between erasure and rewrite of the USB key. Lose power then, and you're out-a-luck. The rsync method doesn't erase anything, except files that were intentionally deleted.

But my prime motivation is that when I shut down my computer, I don't want to wait. The 30 to 60 seconds that it took before was irritating. 5 or 10 seconds is much more palatable.

BTW, if you pass a "-v" argument to rsync, it reports on what it did. I found it interesting to see the Firefox cache files and all the scripts that Puppy refreshes on every boot.

Post Reply