Gnost - backup, restore, copy NTFS, ext, FAT partitions

[ebisu]

[rcrsn51]

@Everyone: Thank you for testing this new generation of Gnost. For newcomers to Gnost, here are some important concepts.

Gnost is a portable app. You install and run it from the drive/partition where the image files are located. So when using a USB hard drive, the program comes along as you move to different machines. If you boot a target Windows machine off a Puppy CD, you don't need to install Gnost - it's already there on the USB drive.

Gnost only works with files in the current directory. Backup image files are too important to get lost or mixed up. You need to know exactly where your files are. That's why you ONLY run Gnost from within the destination directory by clicking on the Gnost program icon. Running Gnost from /opt/gnost3 or from the Puppy menu would create an image file INSIDE your Puppy save file/folder. You don't want to put a multi-GB file there by accident.

So why doesn't Gnost have a file chooser button like other programs? Because after using Gnost for many years, I have concluded that the current method works best. It also adapts nicely to running backups via remote Samba shares.

Regarding Pudd: Pudd doesn't care about the filesystem type of the partition you are backing up. It just uses the dd command to copy everything in the partition, including the unassigned areas that contain no data. That's why it takes hours to back up a modern hard drive.

But Gnost only backs up the areas of the partition that are actually in use. It uses tools that are specific to the filesytem type - NTFS partitions are processed differently from ext partitions. So it's important that both you and the program are aware of the filesytem type in play - hence the radio buttons in the second box. This is crucial if you ever need to restore the image. The target partition MUST match the filesystem type of the original.

I agree that the label on the second box is vague. So I am posting a new version 3.1 that makes this more clear. See the new screenie. Thank you for pointing this out.

However, it seems to me that anyone performing image backups should already have a reasonable idea of what partition types they are dealing with. Just doing a mouseover of the drive icons at the bottom of the screen provides good information. Or you can run Partview from the system tray.

Regarding the List button: because Gnost does not have a file chooser mechanism, I wanted an easy way to select an image file without having to type in the name. Instead, you can copy/paste a filename from the List screen. Some users will complain about this because it's not the conventional method of selecting a file.

But how good are they to restore things? I don't want to mess with my computer and test the restoring functions.

I would never have released Gnost if I didn't have confidence in its ability to perform a restore. I have successfully restored images many times. Gnost uses the same tools as well-known apps like Clonezilla. But no operation working at the hardware level can be guaranteed.

@nilsonmorales: I have changed the two messages at the top of the screen, so they will need re-translating. Sorry about that.

Bill

[nilsonmorales]

rcrsn51
Is ok glad to help, keep the code moving

[rcrsn51]

For experienced users, Gnost v3.2 lets you control the speed/compression of the backup procedure.

[Update] Now that Gnost has switched from gzip to lzop compression, this modification has less value. From Line 4:

Code: Select all

export GZIPARG="-3"		# this is the standard compression for lzop

For greater speed, try "-1"

-----------------------

[Sylvander]

So where is Gnost v3.2?

The latest version available at the 1st post is v3.1
I have that making a backup right now, but I see no difference in operation between it and v3.0

Whoa, now it's at v3.2 [in the 1st post], how does that happen?
Anyway, I'll now download that pet and install ASAP, and run yet another backup.

OK, I'm making a backup right now using v3.2, BUT...
I see no difference in operation to v3.0 and v3.1.
No method evident to "control the speed/compression of the backup procedure".
Backup now complete.
Completed image file size = 3835 M, whereas with v3.0 it was 2997 M [MiB?]
Actually v3.0 backup now shows 3835 M. How come?
Content on sda1 is unchanging.
Seems to me that Gnost is REALLY FAST!
I've grown accustomed to the making image backups being pretty slow [e.g. PUDD seemed unusually slow in the past, on older PC], but not with Gnost on this newer PC.
Perhaps I should re-try PUDD on this newer PC.

[ebisu]

[Sylvander]

1.

rcrsn51explained it. Edit the "gnost" file.

Ouch!
How dumb I am.
I see it now, changed the value to -3 in the gnost file for v3.2.

2 different solutions for different purposes. Pudd creates an image of every sector, bad or good, used or not. This makes a true clone. Use it for forensic analysis of a partition or for data retrieval. There are times when speed is irrelevant.

Thank you for clarifying that.

[gcmartin]

There are as was nicely explained by @Ebisu 2 DIFFERENT types of Backup Programs. There are:

• Image Backup - This creates a "Cloneable" copy of a drive where when it restores, it will restore, datablock by datablock, EXACTLY what you started out with to the same or a new drive. In other words, it REPLICATES what it found, when restored somewhere. This has a VERY important need especially for those OSes that require filesystem to be in a specific place. (Will get to that later) So, image backup has a necessary need and brings a guarantee of data replication.
• Files Backup - This looks at a partition's filesystem and copies files. It is primarily needed to make a copy of the files. And excepting for the Backup programs from vendors, does NOT care about file positioning in the filesystem. Thus, when restored, it will place the files back and does NOT respect the physical locations like an image backup does. It is this reason when any attempt to restore a bootable Windows system requires a repair (thus the original CD and other tools) to "fix" file positioning once a files backup restores a drive.

This thread's title's use of "clone" is a misuse ... unless of course, this Partition File Backup program has been designed to know and respect the locations of files that each version of Windows, for example, requires so that boot integrity is preserved.

In Image Backups, this is unnecessary and the restore will in most every case boot once it is restore to a drive without fanfare.

And yes, PUDD does what it is designed to do for image backup. As I remember, seems that it may also do a files backup of a partition's data as well. These 2, PUDD and Gnost, probably should not be compared as their design approaches are very different.

Most files backup can/does allow the user to restore to a completely different filesystem should they choose. In fact, I have done this many times in the past. Use a files backup program to capture the files from one filesystem, then restore those files to a completely different filesystem.

In most cases, a filesystem backup of a data only partition or a non-bootable partition (no matter the OS) will yield safe restorations of one's files. For a bootable partition/disk, awareness is required. Otherwise user support is needed.

RAID systems has it own nuances.

Hope this brief explanation is helpful.

[Sylvander]

@gcmartin

In the past...
I have used folder/file backup->delete->restore...
As a simple/quick means to defragment the folder/file system [of the windows partition contents].

Could you explain your understanding of how that fits into the whole.

[gcmartin]

I assume you may have done this using a Linux. Correct?

If done in one Windows for another, Windows is smarter than one imagines.

If not, then, you have had success because the critical files for boot integrity were not moved. Not sure how you accomplished your restore.

But, should you have wiped the boot partition in Linux and did a write with the files restore program using Linux, you probably had problems in booting Windows: again, assuming this was done to a Windows boot partition with its boot flag set on a Windows PC containing the Windows boot manager. File restore programs not designed for Windows use, suffer from this problem. The internal data of the files contain all the information of each individual file. We are talking about "where" ... not what which causes the problems.

You can find much on this across the web. This has been discussed and diagnosed in more ways than there are stars in the sky over the past 2 decades. Review.

None of what I have shared says to NOT use this utility. The intent is to expand on couple items mentions by members for our understanding.

This utility does what the author indicates in preserving and allow the preserved to be written out/back.File content integrity will be preserved. And, it takes a simplistic approach. Useful in several situations I know of.

There are many other utilities in and OUT of Puppyland which do drive-partition backup/restores, also over the past few decades. One of the most famous in Linux is Clonezilla which does similar (but more) to what PUDD does: file backup/restore and image backup/restore.

Hope this is helpful.

[ebisu]

[gcmartin]

Yes. Clonezilla features image backup AND offers file backup services. Users choose which service best matches their needs, especially their needs in the event of restoration. PUDD does offer, similarly, the same 2 services to PUP users. Direct disk cloning is available in Clonezilla, IIRC. But, these backup-restore-cloning subsystems has a tract record of decades of tweaking over time as both OSes and storage technologies have changed. And have a long record of data integrity not to mention the many features.

Again, this too, Gnost, is a very good approach in providing a simple solution to this community for capture of the files on a partition and restoring the files.

Keep up the good work, Bill. I'm sure you will advance this package as you see fit.

[Sylvander]

@gcmartin

1. "I assume you may have done this using a Linux. Correct?"
Correct.
Done in "Puppy->Xfe".

2. "Not sure how you accomplished your restore."
Backup, delete, and restore all done using "Puppy->Xfe".

3. "...problems in booting Windows..."
I had no such problems.
Was Win2000Pro on a FAT32 partition.

4. "...assuming this was done to a Windows boot partition with its boot flag set on a Windows PC containing the Windows boot manager."
Was...
"done to a Windows boot partition with its boot flag set on a Windows PC"...
Not sure about...
"containing the Windows boot manager."
But I expect it did.

5. "...should you have wiped the boot partition in Linux..."
I didn't "wipe" the Win2000Pro boot partition, I just deleted the files.

6. "...and did a write with the files restore program using Linux, you probably had problems in booting Windows..."
Again...
I wrote the files back to the empty [folders/files deleted] Win2000Pro boot partition using Puppy->Xfe and subsequently win2000Pro booted OK.

7. "File restore programs not designed for Windows use, suffer from this problem."
My Win2000Pro didn't suffer any problem; does it perhaps only affect later Windows?

[rcrsn51]

Gnost v3.3 is posted above. It replaces traditional gzip compression with lzop. In limited testing, lzop is much faster with only a small decrease in compression. Most (all?) Puppies have lzop built-in through Busybox.

For advanced users, the GZIPARG variable is set at "-3", which is the lzop default. For higher speed, try "-1".

Testers are welcome. Discussions unrelated to this project are not.

Thank you to the anonymous collaborator who has provided invaluable help with this project.

[ebisu]

[rcrsn51]

Thanks for testing.

sda3 (9.6G total 8G used)
Pudd: 35min 7736M
Gnost: 23min 7759M

That's a surprising result. Since the partition is almost full, you would expect the two methods to be almost equivalent. Gnost should make a slightly smaller image because it stores fewer sectors, but it should take slightly longer because of the additional work it does. I'm guessing that the Gnost tools (ntfsclone, partclone, lzop) are optimized for speed over image size.

[Edit] If Pudd is using gzip-6, then it will definitely take longer and make a smaller compressed image than Gnost, which is using lzop-3.

Of course, most Gnost users will be in the opposite situation. They are backing up a large partition that is only partly full.

I understand your reluctance to use the Restore function. But you can test it by restoring onto an unused partition of a USB drive.

For small partitions I feel safer with Pudd

People always feel safer using the tools that they know. But to suggest that dd is safer is to imply that the millions of users of products like Acronis or Clonezilla are using tools that are less safe.

Bill

[rcrsn51]

Gnost v3.4 has a new Space function. It identifies the amount of space being used in the selected partition (before compression to an image file) and the amount of space available in the current location for saving images. These values come from the "df" command and are approximate.

[ebisu]

[bobc]

Lets say I have a linux partition ext4 on SDB5 on machine A and I want to restore it to SDA9 on machine B. I then want to use Grub 2 (which is installed on that partition) to install and update-grub to rebuild the boot menu to add it so I can boot from it?

I have tried this with pudd and gparted, and am able to get the partition successfully onto machine B, I believe.

Then I do a few mount commands, chroot to it, and then run the grub-install and update-grub, which seem to work ok, but it doesn't add the partition SDA9 to the boot menu. It was bootable as SDB5 on machine A, but the same grub 2.0 doesn't find it as SDA9 on machine B.

Since machine B isn't important, if I used your program, I could test the restore there if I could get it to boot.

Any suggestions? Any guess what I missed?

[comcoco]

On this topic, it is stated that gnost 3 is available. I can not find how to get it on the forum. Please help.