Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 03 Sep 2014, 03:14
All times are UTC - 4
 Forum index » House Training » HOWTO ( Solutions )
How to back up a hard disk to another machine
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [17 Posts]   Goto page: 1, 2 Next
Author Message
jeffrey

Joined: 16 Jan 2006
Posts: 162
Location: Christchurch, New Zealand

PostPosted: Wed 02 May 2007, 02:04    Post subject:  How to back up a hard disk to another machine  

I am considering how to make backups of some old PCs that run Linux.
My current favoured option is to use a Live CD (why not Puppy Linux?) to boot from, then use NFS to mount the backup file system on the remote machine, and then simply back up the local disk to the remote machine.
I've tested this out with a Fedora Core 5 installation's /boot partition, which is about 100MB, of which only 20MB is used. I copied it to the remote machine, which was fine. Then I tried again with compression, which reduced the file size from 100MB down to 63MB. Realising that the unused 80MB in the file system is probably full of garbage, I wrote zeros to it, and then copied it again with compression, this time ending up with a mere 13MB file. This seems pretty simple and although it's a manual process, with the disk-zeroing step it is probably more efficient than any automated backup alternative out there. (It wouldn't be hard to automate it, but I haven't time for that at present.)
The steps that I followed to do this (with a standard Puppy Linux 2.16) are:
1. On the remote machine, say monster, allow a directory, say /backups, to be writeable by NFS by adding "/backups (rw)" to /etc/exports and restarting the nfs service.
2. On the local machine mount the NFS disk with "mount -o rw,nolock monster:/backups /mnt/data". The nolock avoids the 30s timeout caused by mount looking for portmap (according to delboy711, post 12523).
3. Wipe the unused 80MB of /dev/hda1 with zeros by mounting it as /mnt/hda1 with the standard Pmount tool, then issue "dd if=/dev/zero of=/mnt/hda1/filler-to-write-zeros-to-disk bs=1024k count=80" and "rm /mnt/hda1/filler-to-write-zeros-to-disk" to release the disk space now that it has been wiped.
4. Copy /dev/hda1 from the local machine, say scary, to the backups directory on the remote machine with "gzip < /dev/hda1 | dd of=/mnt/data/scary-dev-hda1.gz"
And that's all.
To restore it use "gzip -d" in the opposite direction, but I haven't tested this and even if I had I wouldn't guarantee that it will work for you, so don't complain to me if it doesn't work! Experiment on an expendable disk first.
Of course, I'm not actually interested in /dev/hda1, but really in /dev/hda in its entirety. What this experiment tells me is that I am probably getting 100% compression on empty disk space once I've written zeros to it. I would have to do the same to hda2 before backing it up because its the / file system while hda1 is merely the small /boot file system. But the same approach should work there too. I'm not sure about writing zeros to the swap partition (hda3 in my case think) because Puppy may be smart enough to be using it!
I hope that this will be of interest to someone.
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11092
Location: The Peoples Republic of California

PostPosted: Wed 02 May 2007, 11:28    Post subject:  

Interesting.

You can turn your swap partition off, then write zeros.

swapoff /dev/???

Of course you'll get optimal compression if you copy free space which contains zeros.

I'm going to try #4 and see how it works, I might use it to replace my present backup routines.
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11092
Location: The Peoples Republic of California

PostPosted: Wed 02 May 2007, 11:52    Post subject:  

I started the backup with a slight variation, while watching the screen, and thinking, it occurred to me that I might have problems restoring it.

gzip < /dev/hda1 | dd bs=8192 of=/mnt/hda8/hda1.gz

128273+1 records in
128273+1 records out

That should be 9 MB of compressed data.

Seems to have worked fine.

--------------

Restoring theory I'll have to think about and do some R&D

Easy way to wipe free space for better compression theory proposal:

# umount -a
# dd if=/dev/zero of=/zerofile.tmp
# sync
# rm /zerofile.tmp
# sync


----------
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11034
Location: Arizona USA

PostPosted: Wed 02 May 2007, 15:03    Post subject:  

Jeffrey, as a How-To, this leaves too much as an exercise for the reader. Confused See this post. Did you install nfs in Puppy? If so, how?
Back to top
View user's profile Send private message 
valpy

Joined: 18 Apr 2007
Posts: 67
Location: Looking at the tapestry

PostPosted: Wed 02 May 2007, 16:36    Post subject:  

The Pudd utility in Puppy is useful for backups (Start->Utility->Pudd in 2.15, but it's available at least from 2.13 onwards). I have used it, (though not over NFS), but if you're creating a backup file for a disk or set of partitions I think it should work over NFS.

Follow the prompts - it will use dd, writing zeros to the unused parts of the disks/partitions, and will then compress the image as a .gz file (all from the gui).

Pudd can restore images too - a very nice simple backup and restore method.
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11092
Location: The Peoples Republic of California

PostPosted: Thu 03 May 2007, 00:09    Post subject:  

I found the following on the Internet

Creating a hard drive backup image

# dd if=/dev/hda | gzip >/mnt/hdb1/system_drive_backup.img.gz

Here dd is making an image of the first harddrive, and piping it through the gzip compression program. The compressed image is then placed in a file on a separate drive. To reverse the process:

[Restore hard drive from image]

# gzip -dc /mnt/hdb1/system_drive_backup.img.gz | dd of=/dev/hda

Here, gzip is decompressing (the -d switch) the file, sending the results to stdout (the -c switch), which are piped to dd, and then written to /dev/hda.

source:

http://wiki.linuxquestions.org/wiki/Dd#Creating_a_hard_drive_backup_image
Back to top
View user's profile Send private message 
jeffrey

Joined: 16 Jan 2006
Posts: 162
Location: Christchurch, New Zealand

PostPosted: Thu 03 May 2007, 02:11    Post subject:  

Apologies for my poor post. It isn't bullet-proof, I'm sure. I made two particular mistakes:
1. Flash, I didn't realise there were How-To standards that I've fallen short of. Sorry about that. Are these published? Looking at some other HowTos I'm not sure where I'm deficient (apart from including step-by-step restore instructions), but re-reading my instructions I can see that they’re not as clear as they could be. The commands need to be run from a console window (ie from rxvt). NFS must be installed, configured (with a writeable directory), and running on the remote machine (in my case a Fedora Core 5 Linux machine). Puppy, running on the local machine, simply uses its existing ‘mount’ command to mount that remote NFS file system. So Puppy does not need NFS installed for this backup strategy to work. I should also have said that Puppy should be run entirely in RAM so that the hard drive is not mounted during the backup.
2. valpy, I hadn't noticed Pudd (which is quite embarrassing). That’s an excellent piece of work and nearly does all that I need as it stands, but not quite. It has a backup-to-remote-machine option which uses netcat/nc (already present on my remote Fedora machine and an amazingly simple tool that I was ignorant of), but since my remote machine already has NFS I’ll just mount the NFS remote file-system on my local machine as above and get Pudd to write to that mount point as if it is a local directory. At least with Puppy 2.14 (which is what I’m using today), Pudd does not write zeros to file systems before backing them up, so my instructions for that are still valid and can result in a major saving of space and transfer time.

Bruce B, thanks for the “swapoff /dev/hda3” command. The “dd if=/dev/hda | gzip” approach that I used and you found on the Internet is also what Pudd uses. Note that your clearing of unused file space would work but I would never fill a running file system which is what you seem to be are proposing. Instead, from Puppy running in RAM, I would mount each of the hda partitions one by one, fill the unused space with zeros, and then release the space again. To do this for hda1 use “mount /dev/hda1 /mnt/hda1; dd if=/dev/zero of=/mnt/hda1/filler.tmp; sync; rm =/mnt/hda1/filler.tmp; umount /dev/hda1”, etc.

More precise figures for my hda1 disk space are 98MB total, 8MB used (not 20 as I had thought), and 84MB free (with all figures rounded down and ignoring the space overheads used by the inode table, etc).

In defence of my command-line backup instructions (as opposed to a utility such as Pudd), many users (including myself) would like to write a script to perform the backup. In fact I don’t really care if I use Puppy Linux, Feather Linux, Knoppix, a bootable DOS floppy (now there’s a challenge!), or whatever, as long as I can back up a local disk image to a remote NFS file system mounted on the local machine.. Portability is a great comfort, and the command line approach using standard UNIX utilities is the most portable solution that I can think of.

To back up an entire disk, eg hda, Pudd looks quite sufficient but you could save a lot of space and time by writing zeros to each partition first as per my instructions 3 paragraphs back. In my case I did this for hda1 and hda2. If I wanted to clear the swap partition and perform the backup without a swap partition I would use “swapoff /dev/hda3; dd if=/dev/zero of=/dev/hda3”. This would be devastating on an ext3 file system because it wipes the entire partition, which is probably OK for the swap partition but not for any file system that you ever want to use again! To reiterate, this dd command wipes the whole partition, not just unused space, so be careful that your hda3 is actually a swap partition, and even then I haven’t test this so it may be best left alone.

If this post is too technical for the How To section please move it elsewhere, but it would be good not to delete it since there may be others out there like me who want to do this sort of thing efficiently from the command line…
Back to top
View user's profile Send private message 
valpy

Joined: 18 Apr 2007
Posts: 67
Location: Looking at the tapestry

PostPosted: Thu 03 May 2007, 02:40    Post subject:  

If you use Pudd, there is a dialogue at the point of creating the image which does give you the option of writing zeros to unused parts of the image - or you can do it yourself.
Back to top
View user's profile Send private message 
valpy

Joined: 18 Apr 2007
Posts: 67
Location: Looking at the tapestry

PostPosted: Thu 03 May 2007, 02:42    Post subject:  

Should have said, this is in 2.13 - haven't tried 2.14
Back to top
View user's profile Send private message 
valpy

Joined: 18 Apr 2007
Posts: 67
Location: Looking at the tapestry

PostPosted: Thu 03 May 2007, 03:16    Post subject:  

Just tried Pudd in 2.14, it does offer the option to write zeroes.

There is a final dialogue before the image is created

"Puppy Universal DD: optimise compression

/dev/sda6 will be copied to /root/myfile.img.gz, compressed with gzip.
Compression may be greatly improved if the unused part of /dev/sda6 is zeroised. This involves temporarily mouning it on /mnt/tmp, writing zeroes to the unused areeas, then unmounting it.
Would you like to do this size optimisation?"
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11034
Location: Arizona USA

PostPosted: Thu 03 May 2007, 10:29    Post subject:  

Jeffrey, I only delete spam, obvious duplicate posts, posts with objectionable language that otherwise have no redeeming value, etc.. You're safe. Smile

It can be pretty hard to see which forum a post like yours belongs in. I thought it had the makings of a useful How-To, with just a little more work.

I realize that there are no written standards for a How-To. Like everything else in Puppy, this forum is a work in progress, done by volunteer enthusiasts. Still, it seems obvious to me that a How-To should be self-sufficient, and written so that the average puppian can give it a try. As the forum's index page suggests, How-Tos are meant to be like recipes. Experimental recipes. They might not turn out to work for everyone, but they should be tested at least once by the author and written clearly enough and contain enough information to enable the target audience to assemble the ingredients and try it for themselves. Smile
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11092
Location: The Peoples Republic of California

PostPosted: Thu 03 May 2007, 16:03    Post subject:  

Flash, I posted right below you and now its gone. Did you delete it?
Back to top
View user's profile Send private message 
jeffrey

Joined: 16 Jan 2006
Posts: 162
Location: Christchurch, New Zealand

PostPosted: Thu 03 May 2007, 18:05    Post subject:  

valpy, my apologies about Pudd and wiping unused areas of file systems with zeros. You are quite correct that for a partition (ie file system) it does offer such an option. I missed that. Note that it doesn't do this when backing up an entire drive (which is what I want). I may spend a little time writing an enhancement to Pudd to do that. Pudd is quite an impressive piece of work as it is.

Thank you all for your helpful and patient posts.

I use Puppy Linux at home. It is regarded as a personal toy at work, but with its excellent hardware detection and Pudd that attitude may change...
Back to top
View user's profile Send private message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11034
Location: Arizona USA

PostPosted: Fri 04 May 2007, 00:03    Post subject:  

Bruce B wrote:
Flash, I posted right below you and now its gone. Did you delete it?

No, and I didn't see any post from you other than the one quoted. Anyway, it's been days since I deleted a post.
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11092
Location: The Peoples Republic of California

PostPosted: Fri 04 May 2007, 11:32    Post subject:  

Flash wrote:
Bruce B wrote:
Flash, I posted right below you and now its gone. Did you delete it?

No, and I didn't see any post from you other than the one quoted. Anyway, it's been days since I deleted a post.


Thank you for your reply. I didn't think it was you, but now I know. Once in a while a post doesn't get posted. I might do well to save the text before submitting.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 2 [17 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » HOWTO ( Solutions )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0930s ][ Queries: 11 (0.0050s) ][ GZIP on ]