Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 27 Aug 2014, 19:16
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
Reduce & speedup shutdown writes to USB with rsync
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [4 Posts]  
Author Message
billstclair

Joined: 26 Feb 2006
Posts: 106
Location: Upstate New York

PostPosted: Tue 14 Mar 2006, 00:29    Post subject:  Reduce & speedup shutdown writes to USB with rsync  

I did this in Puppy 1.0.8. It will likely work, with slight script changes, in earlier versions. Puppy 2.0 works differently in this regard so probably doesn't need any such patch.

I'm running Puppy (1.0.8) from a 1 gig USB key. I let Puppy do the USB install for me. I noticed that shutdown took a long time "writing back to the pupxxx file on the Flash drive from ramdisk", making the USB access light flash the entire time. I found the responsible code in /etc/rc.d/rc.reboot:

Code:
 rm -fr $PUPMNTPOINT/*
 #...heh, heh, this leaves behind hidden files/dirs, so do this...
 rm -fr $PUPMNTPOINT/.[a-zA-Z0-9]*
 sync
 cp -af /root/. $PUPMNTPOINT
 sync


This deletes all files from the pup100 file system on the USB key and copies the versions in the ramdisk back to there. If you have lots of extras loaded, this is a fairly large amount of data. My installation, for example, with Emacs, Firefox, and a few other smaller packages, is about 146 megs.

It's really only necessary to copy files that have changed. rsync is a standard Linux program that excels at doing exactly that. This HOWTO describes installing rsync and changing Puppy's startup and shutdown scripts to use it instead of cp to save the changes to /root.

rsync is not part of the standard Puppy installation. It isn't even available, yet, as a dotpup.

To build it, you need to get usr_devx.sfs, put it in your HOME directory (the same place as the pupxxx file), and reboot. Then download the rsync source from http://www.samba.org/rsync/. The latest stable version at this writing is rsync-2.6.7.tar.gz. Put that file somewhere, e.g. /root/my-documents. Then do the following:

Code:
cd /root/my-documents
tar -xzf rsync*.gz
cd rsync*
./configure
make
make install


This puts an executable file, rsync, in /usr/local/bin.

Alternatively, you can download the attached rsync.gz, gunzip it, and mv it to /usr/local/bin. That should work for 1.0.8, where I compiled it, but may not work for other versions of Puppy.

Once you have rsync installed, you can back up /root with the following commands:

Code:
rsync -a --delete /root/ /mnt/pupxxx
sync


The rsync command makes /mnt/pupxxx a copy of /root, copying as little as possible to make it so. sync flushes buffers to the storage medium. You can easily put this into a script file and bind it to a desktop icon to allow incremental saves whenever you want. Or use the crontab to schedule it at regular intervals.

To change Puppy's startup and shutdown scripts to use rsync for the copy:

Code:
cd /root
cp /etc/rc.d/rc.reboot rc.reboot.rsync


Now edit rc.reboot.rsync, and change the code listed above to:

Code:
# rm -fr $PUPMNTPOINT/*
 #...heh, heh, this leaves behind hidden files/dirs, so do this...
# rm -fr $PUPMNTPOINT/.[a-zA-Z0-9]*
 sync
# cp -af /root/. $PUPMNTPOINT
 /usr/local/bin/rsync -a --delete /root/ $PUPMNTPOINT
 sync


Finally, add the following to the end of /etc/rc.d/rc.local:

Code:
# Use rsync to copy /root/ to /mnt/pupxxx
cp -f /root/rc.reboot.rsync /etc/rc.d/rc.reboot
cp -f /root/rc.reboot.rsync /tmp/rc.reboot


You can't just edit rc.reboot in place in /etc/rc.d because Puppy overwrites it with the virgin copy on every boot. The /tmp copy is the one that is actually used at shutdown or reboot time.

The first shutdown after making these changes will do the copy as usual. Subsequent shutdowns will be much quicker and will write much less to your USB key.
rsync.gz
Description 
gz

 Download 
Filename  rsync.gz 
Filesize  267.97 KB 
Downloaded  290 Time(s) 

Last edited by billstclair on Tue 14 Mar 2006, 00:44; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
billstclair

Joined: 26 Feb 2006
Posts: 106
Location: Upstate New York

PostPosted: Tue 14 Mar 2006, 00:37    Post subject:  

One caveat. When you update your Puppy, you'll need to remove or comment out the changes to /etc/rc.d/rc.local before doing the update, then remake /root/rc.reboot.rsync by editing the updated version of /etc/rc.d/rc.reboot, and restore the rc.local lines. Or maybe 1.0.9 will break this...

I haven't checked how Barry does the incremental backups in Puppy 2.0. That may benefit from rsync as well...
Back to top
View user's profile Send private message Visit poster's website 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11024
Location: Arizona USA

PostPosted: Tue 14 Mar 2006, 11:24    Post subject:  

Incremental backups are a good idea to save space and time, but it seems like it would also be a good idea to include an occasional "full backup", like MPEG saves a full frame every so often.
Back to top
View user's profile Send private message 
billstclair

Joined: 26 Feb 2006
Posts: 106
Location: Upstate New York

PostPosted: Tue 14 Mar 2006, 12:02    Post subject:  

I don't understand how the concept of "full backup" applies here. The RAM disk is copied from the pup100 file at startup. Then some incremental changes are made to the RAM disk. These need to be copied back to persistent storage. rsync ensures that the pup100 image exactly matches the RAM disk. Barry's erase and copy method DOES improve locality, which might matter if you were writing to a disk, but FLASH RAM has no locality issues that I know of.

Personally, I don't get a warm fuzzy feeling from knowing that my system exists only in RAM during the time between erasure and rewrite of the USB key. Lose power then, and you're out-a-luck. The rsync method doesn't erase anything, except files that were intentionally deleted.

But my prime motivation is that when I shut down my computer, I don't want to wait. The 30 to 60 seconds that it took before was irritating. 5 or 10 seconds is much more palatable.

BTW, if you pass a "-v" argument to rsync, it reports on what it did. I found it interesting to see the Firefox cache files and all the scripts that Puppy refreshes on every boot.
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 1 [4 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0587s ][ Queries: 13 (0.0058s) ][ GZIP on ]