snap2 rotating snapshot backups for Puppy

Miscellaneous tools
Message
Author
User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#136 Post by lstandish »

I had trouble with posting, so I replied via PM. I fixed that problem so I'm posting the PM interchange here:

Hi,
You found a bug! It turns out that backup exclusions cannot be added if the backup directory is the root directory ('/').

Here's my excuse for overlooking this bug: I never intended for the program to backup the entire filesystem. My idea was that only the user's data files would be backed-up.

Anyway, a bug is a bug, and I'll try to fix it soon. Until the bugfix release comes out, you'll have to specify any directory except '/' if you want to use backup exclusion patterns.

To this, "J" replied via PM:

Cool, but i did not exclude "/", I excluded "/mnt".

Please help me understand "entire file system"-- I just want a snapshot of my puppy install-- essentially the personal save file, but incremental and compressed. That's why I tried to exclude my mounted drives.

Love GUI's that simplify my tasks, but feeling overwhelmed by your myriad options and pages. Just want to do a manual backup before I make significant system changes.

Maybe you could offer a simplified one-click interface? Would be awesome!

Thanks!
J

To which I reply:

Hi again (right this moment I'm fixing this bug in snap2!)

What I meant is that if the backup directory specified is '/' (as it is in this case) then you can't specify any backup exclusions. I understand that you want to exclude your mounted drives from the backup, and that's a good idea.

If you want a backup of your entire Puppy installation in order to recover to a previous state of installed packages, etc. (like Windows' "Recover to a Previous Time"), I'd suggest a one-time backup of the Puppy save file.

However, for regular backups and for access to previous *versions* of files, this is inefficient since the save file has a fixed size no matter how little the content. Also, a Puppy save file backup does not allow snap2/rsync to take advantage of hardlinks to save disk space.

snap2 is intended to back up user data files, not the entire OS. If the OS fails (hard drive failure, etc.), the idea is that you fix things, reinstall the OS, then copy over the backed-up user data files. Or, you can use the backup to recover a file you deleted by mistake, or to revert to a previous version.

Following are my suggestions for using snap2 this way with Puppy:

I would suggest backing up only '/root', which should get all your documents and similar files, since the backup will include all subdirectories. If there are any other directories besides /root where you store files (such as special PETS you've downloaded), add them as "Backup Source Directories" too.

You probably won't have to bother with exclusions if you back up only '/root'.

Your backup backup storage location must be OUTSIDE the Puppy virtual filesystem, so it will probably be a path beginning with '/mnt'. That way, if your Puppy save file is corrupt or Puppy itself has a major problem, your files are accessible from another OS, or by booting Puppy from a CD.

If you have to reinstall puppy, it is a simple matter to copy the backed-up files from the backup storage location to the new Puppy.

Note that this scheme will allow you to recover (from backup) any previous *version* of files. Also, only changed files will actually occupy additional space on the backup drive at the time a "snapshot" is made. Unchanged files at the time of every new snapshot are hard-linked.

However files are NOT compressed. By avoiding compression, they can be accessed and recovered by simply copying with any file manager. The savings of backup storage space is so great with snap2 that you will almost certainly find that compression is unnecessary.

In sum, if you want simple, fast backups, specify '/root' as the "Snapshot Backup Path" on tab one. Follow the 4 steps at the top portion of the snap2 GUI. You can ignore the logs, advanced options, etc., since they're set to reasonable defaults.

However, if your backup disk might sometimes not be mounted at backup time, I suggest you uncheck the "allow no hardlink reference" option on the ADVANCED tab under Miscellaneous Settings, after the first backup. (It MUST remain checked for the first backup!)

User avatar
johnywhy
Posts: 879
Joined: Sat 20 Aug 2011, 14:52

#137 Post by johnywhy »

Yep, I understand the benefits of rsync (snap2). That's why I want to use it :)
...fix things, reinstall the OS, then copy over the backed-up user data files...
You're saying we fix things BEFORE restoring the backup? If i break the OS, I would not want to "fix" it, I would just want to do a complete restore of the whole OS to an earlier state. Also, the way I prefer to use backup is to avoid reinstalling the OS. I've achieved that with a simple drag-copy of the personal save file, and hoped to achieve that with rsync, with the benefit of incremental backups. Is that not possible?
...I would suggest backing up only '/root', which should get all your documents...
I do not store any documents in the root, I put my docs in the mounted drive. I just want to backup puppy settings, desktop, and installed apps. Does /root contain all that?
...if your backup disk might sometimes not be mounted at backup time, I suggest you uncheck the "allow no hardlink reference"...
So, if my backup disk is always mounted at the time I do the backup, including the first time, and I leave the "allow no hardlink reference" option checked each time, then will I get the benefit of hardlinks? (btw, I cannot imagine why i would attempt a backup without my backup disk connected).
...By avoiding compression, they can be accessed and recovered by simply copying with any file manager....
Is that true, if I will ONLY be doing a complete restore of the entire OS? I do not need to access individual files, just the whole thing. I've read that rsync backup with compression is a bit faster than uncompressed, which would be cool. How DO we restore a compressed rsync backup?

Thanks!

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#138 Post by lstandish »

johnywhy wrote: You're saying we fix things BEFORE restoring the backup? If i break the OS, I would not want to "fix" it, I would just want to do a complete restore of the whole OS to an earlier state.
By fix things I mean fix a failed hard drive. Actually as a Linux user I never think of restoring from a backup to fix configuration mistakes, only to recover deleted files, corrupt files, previous versions of files, or to recover from a failed hard drive.
Also, the way I prefer to use backup is to avoid reinstalling the OS. I've achieved that with a simple drag-copy of the personal save file, and hoped to achieve that with rsync, with the benefit of incremental backups. Is that not possible?
Not really. rsync will transfer files quickly by transferring only changed portions. rsync by itself is not "incremental." snap2, by using new backup storage only for changed and new files and allowing a series of "snapshots", can be thought of as "incremental," but if you are backing up a single huge file (Puppy save file) a new copy on disk will be necessary for every backup.
I do not store any documents in the root, I put my docs in the mounted drive. I just want to backup puppy settings, desktop, and installed apps. Does /root contain all that?
'/root' is the Puppy user 'home' directory and would be the default location to store documents and such. If you store them outside of puppy filesystem (on a /mnt/sdd1/ drive for example), that's OK too, and snap2 can always be used to back those up.
So, if my backup disk is always mounted at the time I do the backup, including the first time, and I leave the "allow no hardlink reference" option checked each time, then will I get the benefit of hardlinks? (btw, I cannot imagine why i would attempt a backup without my backup disk connected).
"Allow no hardlink reference" only allows a backup to proceed if there is no previous backup to use as a reference for the hardlinks. This option is used to avoid creating a full, non-hardlinked backup when the backup media is accidentally not connected. This can rather easily happen if the backup media is a USB drive. If your backup drive is always connected, you can safely leave this checked.
"...By avoiding compression, they can be accessed and recovered by simply copying with any file manager...."

Is that true, if I will ONLY be doing a complete restore of the entire OS? I do not need to access individual files, just the whole thing.
I think your best bet would be to use snap2 to backup the Puppy save file. You could use mirror mode to keep only a single backup, if you don't have room for more. Or there might be other backup tools that are better suited to what you want.
I've read that rsync backup with compression is a bit faster than uncompressed, which would be cool. How DO we restore a compressed rsync backup?
You'd have to uncompress it first. You'd need an OS for that, of course! You would boot Puppy from CD, install, then decompress the save file to the right place.

User avatar
johnywhy
Posts: 879
Joined: Sat 20 Aug 2011, 14:52

#139 Post by johnywhy »

as a Linux user I never think of restoring from a backup to fix configuration mistakes, only to recover deleted files, corrupt files, previous versions of files, or to recover from a failed hard drive.
Me too. Who said anything about mistakes? Puppy is not a rock solid platform. I've had to reinstall many times, thus losing all my apps and settings. I now drag copy the PSF before making changes, to easily restore the OS if it breaks, without having to reinstall the OS. But takes time and disk space. Hoped rsync/snap2 would give me a better solution-- sounds like not.
rsync will transfer files quickly by transferring only changed portions.
Sounds great. If "incremental" is the wrong word to use, my bad (although I have seen "incremental" used on more than one discussion of rsync). Are you saying rsync does or does not backup only changed files? Unclear. Also, you're saying snap2 is more "incremental" than rsync, or saves space better than rsync?
if you are backing up a single huge file (Puppy save file) a new copy on disk will be necessary for every backup.
The puppy file system directory IS the puppy save file, is it not? Isn't backing up the file system equivalent to backing up the PSF? I'm no puppy expert. is reinstalling puppy the only way to benefit from an rsync "incremental" backup?
/root' is the Puppy user 'home' directory and would be the default location to store documents and such. If you store them outside of puppy filesystem (on a /mnt/sdd1/ drive for example), that's OK too, and snap2 can always be used to back those up.
nice, but not trying to backup my docs. Trying to backup the OS. Again, does /root contain puppy desktop, configuration, and installed apps? Sorry for my ignorance.
"Allow no hardlink reference" only allows a backup to proceed if there is no previous backup to use as a reference for the hardlinks.
then wouldn't that mean that leaving it checked would prevent backup attempts after the first one? Sounds like what you're saying.
I think your best bet would be to use snap2 to backup the Puppy save file.
What's the advantage of that, over drag copy?
You'd have to uncompress it first. You'd need an OS for that
Thats why I dual boot.

Sounds like snap2/rsync cannot be used for "incremental" backups of the OS :(

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#140 Post by lstandish »

1. Ok, I mean I don't use a backup program to recover a broken OS. As a sys admin, I find Debian suits me better than Puppy, and it never breaks. I promote Puppy for those whose computer use is less technical.

2. rsync alone is not incremental, although it can be used to back up only new/changed files, if that's what you mean. That will result in a single backup. snap2 is "more" incremental because it allows a series of backups.

3. "The puppy file system directory IS the puppy save file, is it not?" Yes and no. It is an "overlay" containing all the files that have changed or been added to the puppy RAM-based filesystem. Puppy RAM filesystem + save file = current state of filesystem. Reverting to a previous version of the save file will do what you want.

4. Lloyd: "Allow no hardlink reference" only allows a backup to proceed if there is no previous backup to use as a reference for the hardlinks."
then wouldn't that mean that leaving it checked would prevent backup attempts after the first one? Sounds like what you're saying.
Golly, no. I'll put it another way: check it to *allow* a backup that will take up as much backup storage as the original files, in case a previous backup is not found. Uncheck it to *force* it to use hardlinks, meaning it will not do a backup at all if it can't locate a previous backup to use for the hardlinks. If unclear, just leave it checked and don't worry. It is only a convenience feature.

5. The advantage of snap2 for backing up a Puppy save file would be that only changed portions need be transferred, but it would take just as much backup storage as a regular copy. Also snap2 can automatically keep several older versions for you, automatically deleting the oldest. Maybe this is not enough advantage to make it worthwhile.

6. "Sounds like snap2/rsync cannot be used for "incremental" backups of the OS"
A series of save file backups can be automatically stored: yes. Incremental: No.

I finished fixing the bug in snap2 that caused backup exclusions not to work for the root directory ('/'), but some testing will necessary because the changes were not trivial.

User avatar
johnywhy
Posts: 879
Joined: Sat 20 Aug 2011, 14:52

#141 Post by johnywhy »

Thanks for reply and info!

Anyone know of an "incremental" or differential backup method that can work with a single file? (such as the PSF, for example :). To backup just the changed parts of an individual file?

The PSF is already compressed, right?

Cheers!

User avatar
don570
Posts: 5528
Joined: Wed 10 Mar 2010, 19:58
Location: Ontario

#142 Post by don570 »

Anyone know of an "incremental" or differential backup method that can work with a single file?
There's the 'diff' command and then you use the 'patch' command

A bit complicated to use :cry:

For instance when a patch is applied to source code by a
programmer. This is the command I used to
apply a patch to mtpaint before compiling.

Code: Select all

patch --input=/mnt/sdc1/tablet_test.patch --directory=/root/mtpaint3.34.56/src

http://linux.about.com/library/cmd/blcmdl1_diff.htm

http://linux.about.com/od/commands/l/blcmdl1_patch.htm

_________________________________________________
Last edited by don570 on Sat 26 May 2012, 16:55, edited 2 times in total.

User avatar
johnywhy
Posts: 879
Joined: Sat 20 Aug 2011, 14:52

#143 Post by johnywhy »

That rocks. Cannot wait to try it! Thanks!

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

rdiff-backup

#144 Post by lstandish »

I think rdiff-backup would also work to keep one current copy of the puppy save file, and "binary diffs" that would allow recovery of a previous save file. From the program's features page:

"Space efficient: Suppose you have a large database file that changes a little bit every day. A normal incremental backup would keep saving copy after copy of this database, wasting a lot of space. rdiff-backup uses librsync, which implements the same efficient diffing algorithm that rsync uses. It works on binary files as well as text, so only a fraction of the data in your database would be saved in each incremental backup."

(As it says above, snap2 would store "copy after copy" of any large file - like the puppy save file - that changes.)

I found this here: http://www.nongnu.org/rdiff-backup/features.html

I don't know whether or not there is a Puppy pet, but I think rdiff-backup can be installed easily on Puppy.

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

Question

#145 Post by peebee »

A quick question if I may....

I've got Snap2 backing up from my working disk sda1 to my backup disk sdb1 at 19.00 each day.

All seems to be fine except that in my backup directory there is a directory for each day/backup which seems to contain just an empty /mnt directory - see screenie attached.

Can I safely delete all these daily directories??

Many thanks
Cheers
peebee
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#146 Post by lstandish »

With snap2, you can always safely delete any backup, unless it contains data you need. However, note that if you have empty daily.x directories, you probably have a misconfiguration. Check your latest backup (recent.1). If it is contains what you expect, you can probably conclude that your current configuration is OK.

In your screenshot I see a number of Backup_<date> directories. Those were created by a different backup program, right?

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#147 Post by peebee »

lstandish wrote:In your screenshot I see a number of Backup_<date> directories. Those were created by a different backup program, right?
Hi lstandish

Thanks for the reply - no other backup program is in use - all those directories - 1 per day at 19.00 since I started using snap2 - are created by snap2 - they appear to be empty except for an empty /mnt directory in each.

The daily.x directories have the expected content.

Thanks
peebee
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#148 Post by lstandish »

Something is fishy - snap2 doesn't create any directories called "Backup_<date>", not even temporary directories. Can you send me your /.snap2/default.set directory (compressed), maybe via PM?

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#149 Post by peebee »

lstandish wrote:Something is fishy
OOOPS!

Profound apologies - my bad - looks like there was a pschedule task lurking from when I tried pbackup.

Really sorry to have bothered you!!!

Thanks
Peebee
Attachments
capture7699.png
(20.72 KiB) Downloaded 473 times
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

User avatar
Pete22
Posts: 264
Joined: Fri 08 May 2009, 22:59
Location: Utah, USA

Does snap2 work on ext 4

#150 Post by Pete22 »

Hello lstandish,

I am going to bite the bullet and do what I should have been doing all along.
That is backing up. My system is Lhp 64 on ext.4
Does Snap2 work on ext.4 or should I reformat and go with ext.3?

Thanks so much for a nice program.


Pete

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#151 Post by lstandish »

Hi Pete,
ext 4 should be fine. snap2 is a frontend to rsync, so I think it will work on any file system that support hard links, which is all Linux filesystems, I believe.

User avatar
Pete22
Posts: 264
Joined: Fri 08 May 2009, 22:59
Location: Utah, USA

Thanks

#152 Post by Pete22 »

That's wonderful news.

My next step is to get it setup and working.


Pete

Jasper

#153 Post by Jasper »

Hi,

I use Snap2 with ext4. It's superb, reliable and fast - it even reports how fast - the incremental option is usually >100 times faster than a full backup.

My regards

User avatar
lstandish
Posts: 126
Joined: Fri 06 Jun 2008, 13:22

#154 Post by lstandish »

I'm glad you've found the program useful, Jasper.

Do you (or anyone else who reads this forum post) use snap2 to backup to *remote* media via Internet? This would normally be to a VPS (virtual private server) or a dedicated sever.

I ask because in the years since I wrote snap2, I've never heard that anyone (besides myself) has used the backup-to-remote-server option.

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#155 Post by peebee »

Hi lstandish

I'm a happy user of snap2 on Slacko 5.3.3 at the moment.

I've been looking into upgrading to a more recent Puppy - and the one that seems to meet my needs is the LxPup variant of Precise.

I've been trying to get Snap2 running today and have run into problems, so I'm wondering if you can advise?

On LxPup (and also on Upup Precise that LxPup is built on) I get gui problems.

I've spent all day looking at your gtkdialog code but have come to the conclusion that I'm unable to work out what's going wrong.

What I've done is replace all occurrences of GTKDIALOG with GTKDIALOG -d in /usr/bin/snap2 and then run snap2 in a terminal - the output is:

Code: Select all

sh-3.00# snap2
widget_entry_refresh(): <label> not implemented for this widget.
widget_entry_refresh(): <label> not implemented for this widget.
widget_entry_refresh(): <label> not implemented for this widget.
widget_combobox_input_by_command(): <input> not implemented for this widget.
/usr/bin/snap2: eval: line 660: syntax error near unexpected token `<'
/usr/bin/snap2: eval: line 660: `<window title="snap2 v4.25 | User root | Settings: default.set"><hbox><vbox><text use-markup="true"><label>"<b>Backup Configuration in 4 Easy Steps</b>'
I'm wondering if the multi-line text message in ADD_DIALOG is throwing it?

The version of GTKDIALOG in LxPup is:
sh-3.00# gtkdialog -v
gtkdialog version 0.8.3 r473M (C) 2003-2007 Laszlo Pere, 2011-2012 Thunor
Built with additional support for: Glade.
Hope we can work out whats wrong and get it going
Cheers
peebee

p.s. Snap2 4.25 works OK in Precise 5.4.1 with gtkdialog version 0.8.0 (C)

Snap2 4.25 run from terminal in Slacko 5.3.7.7 gives similar "not implemented for this widget" errors but works. gtkdialog version 0.8.2 r455M (C) Built with additional support for: Glade, VTE.
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

Post Reply