Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 19 Dec 2014, 21:09
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
snap2 rotating snapshot backups for Puppy
Post new topic   Reply to topic View previous topic :: View next topic
Page 10 of 11 [158 Posts]   Goto page: Previous 1, 2, 3, ..., 8, 9, 10, 11 Next
Author Message
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 24 May 2012, 20:36    Post subject:  

I had trouble with posting, so I replied via PM. I fixed that problem so I'm posting the PM interchange here:

Hi,
You found a bug! It turns out that backup exclusions cannot be added if the backup directory is the root directory ('/').

Here's my excuse for overlooking this bug: I never intended for the program to backup the entire filesystem. My idea was that only the user's data files would be backed-up.

Anyway, a bug is a bug, and I'll try to fix it soon. Until the bugfix release comes out, you'll have to specify any directory except '/' if you want to use backup exclusion patterns.

To this, "J" replied via PM:

Cool, but i did not exclude "/", I excluded "/mnt".

Please help me understand "entire file system"-- I just want a snapshot of my puppy install-- essentially the personal save file, but incremental and compressed. That's why I tried to exclude my mounted drives.

Love GUI's that simplify my tasks, but feeling overwhelmed by your myriad options and pages. Just want to do a manual backup before I make significant system changes.

Maybe you could offer a simplified one-click interface? Would be awesome!

Thanks!
J

To which I reply:

Hi again (right this moment I'm fixing this bug in snap2!)

What I meant is that if the backup directory specified is '/' (as it is in this case) then you can't specify any backup exclusions. I understand that you want to exclude your mounted drives from the backup, and that's a good idea.

If you want a backup of your entire Puppy installation in order to recover to a previous state of installed packages, etc. (like Windows' "Recover to a Previous Time"), I'd suggest a one-time backup of the Puppy save file.

However, for regular backups and for access to previous *versions* of files, this is inefficient since the save file has a fixed size no matter how little the content. Also, a Puppy save file backup does not allow snap2/rsync to take advantage of hardlinks to save disk space.

snap2 is intended to back up user data files, not the entire OS. If the OS fails (hard drive failure, etc.), the idea is that you fix things, reinstall the OS, then copy over the backed-up user data files. Or, you can use the backup to recover a file you deleted by mistake, or to revert to a previous version.

Following are my suggestions for using snap2 this way with Puppy:

I would suggest backing up only '/root', which should get all your documents and similar files, since the backup will include all subdirectories. If there are any other directories besides /root where you store files (such as special PETS you've downloaded), add them as "Backup Source Directories" too.

You probably won't have to bother with exclusions if you back up only '/root'.

Your backup backup storage location must be OUTSIDE the Puppy virtual filesystem, so it will probably be a path beginning with '/mnt'. That way, if your Puppy save file is corrupt or Puppy itself has a major problem, your files are accessible from another OS, or by booting Puppy from a CD.

If you have to reinstall puppy, it is a simple matter to copy the backed-up files from the backup storage location to the new Puppy.

Note that this scheme will allow you to recover (from backup) any previous *version* of files. Also, only changed files will actually occupy additional space on the backup drive at the time a "snapshot" is made. Unchanged files at the time of every new snapshot are hard-linked.

However files are NOT compressed. By avoiding compression, they can be accessed and recovered by simply copying with any file manager. The savings of backup storage space is so great with snap2 that you will almost certainly find that compression is unnecessary.

In sum, if you want simple, fast backups, specify '/root' as the "Snapshot Backup Path" on tab one. Follow the 4 steps at the top portion of the snap2 GUI. You can ignore the logs, advanced options, etc., since they're set to reasonable defaults.

However, if your backup disk might sometimes not be mounted at backup time, I suggest you uncheck the "allow no hardlink reference" option on the ADVANCED tab under Miscellaneous Settings, after the first backup. (It MUST remain checked for the first backup!)
Back to top
View user's profile Send private message 
johnywhy

Joined: 20 Aug 2011
Posts: 353

PostPosted: Thu 24 May 2012, 22:18    Post subject:  

Yep, I understand the benefits of rsync (snap2). That's why I want to use it Smile
Quote:

...fix things, reinstall the OS, then copy over the backed-up user data files...

You're saying we fix things BEFORE restoring the backup? If i break the OS, I would not want to "fix" it, I would just want to do a complete restore of the whole OS to an earlier state. Also, the way I prefer to use backup is to avoid reinstalling the OS. I've achieved that with a simple drag-copy of the personal save file, and hoped to achieve that with rsync, with the benefit of incremental backups. Is that not possible?
Quote:

...I would suggest backing up only '/root', which should get all your documents...

I do not store any documents in the root, I put my docs in the mounted drive. I just want to backup puppy settings, desktop, and installed apps. Does /root contain all that?
Quote:

...if your backup disk might sometimes not be mounted at backup time, I suggest you uncheck the "allow no hardlink reference"...

So, if my backup disk is always mounted at the time I do the backup, including the first time, and I leave the "allow no hardlink reference" option checked each time, then will I get the benefit of hardlinks? (btw, I cannot imagine why i would attempt a backup without my backup disk connected).
Quote:

...By avoiding compression, they can be accessed and recovered by simply copying with any file manager....

Is that true, if I will ONLY be doing a complete restore of the entire OS? I do not need to access individual files, just the whole thing. I've read that rsync backup with compression is a bit faster than uncompressed, which would be cool. How DO we restore a compressed rsync backup?

Thanks!
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Thu 24 May 2012, 22:54    Post subject:  

johnywhy wrote:
You're saying we fix things BEFORE restoring the backup? If i break the OS, I would not want to "fix" it, I would just want to do a complete restore of the whole OS to an earlier state.

By fix things I mean fix a failed hard drive. Actually as a Linux user I never think of restoring from a backup to fix configuration mistakes, only to recover deleted files, corrupt files, previous versions of files, or to recover from a failed hard drive.
Quote:

Also, the way I prefer to use backup is to avoid reinstalling the OS. I've achieved that with a simple drag-copy of the personal save file, and hoped to achieve that with rsync, with the benefit of incremental backups. Is that not possible?

Not really. rsync will transfer files quickly by transferring only changed portions. rsync by itself is not "incremental." snap2, by using new backup storage only for changed and new files and allowing a series of "snapshots", can be thought of as "incremental," but if you are backing up a single huge file (Puppy save file) a new copy on disk will be necessary for every backup.
Quote:

I do not store any documents in the root, I put my docs in the mounted drive. I just want to backup puppy settings, desktop, and installed apps. Does /root contain all that?

'/root' is the Puppy user 'home' directory and would be the default location to store documents and such. If you store them outside of puppy filesystem (on a /mnt/sdd1/ drive for example), that's OK too, and snap2 can always be used to back those up.
Quote:

So, if my backup disk is always mounted at the time I do the backup, including the first time, and I leave the "allow no hardlink reference" option checked each time, then will I get the benefit of hardlinks? (btw, I cannot imagine why i would attempt a backup without my backup disk connected).

"Allow no hardlink reference" only allows a backup to proceed if there is no previous backup to use as a reference for the hardlinks. This option is used to avoid creating a full, non-hardlinked backup when the backup media is accidentally not connected. This can rather easily happen if the backup media is a USB drive. If your backup drive is always connected, you can safely leave this checked.
Quote:

"...By avoiding compression, they can be accessed and recovered by simply copying with any file manager...."

Is that true, if I will ONLY be doing a complete restore of the entire OS? I do not need to access individual files, just the whole thing.

I think your best bet would be to use snap2 to backup the Puppy save file. You could use mirror mode to keep only a single backup, if you don't have room for more. Or there might be other backup tools that are better suited to what you want.
Quote:

I've read that rsync backup with compression is a bit faster than uncompressed, which would be cool. How DO we restore a compressed rsync backup?

You'd have to uncompress it first. You'd need an OS for that, of course! You would boot Puppy from CD, install, then decompress the save file to the right place.
Back to top
View user's profile Send private message 
johnywhy

Joined: 20 Aug 2011
Posts: 353

PostPosted: Thu 24 May 2012, 23:30    Post subject:  

Quote:
as a Linux user I never think of restoring from a backup to fix configuration mistakes, only to recover deleted files, corrupt files, previous versions of files, or to recover from a failed hard drive.

Me too. Who said anything about mistakes? Puppy is not a rock solid platform. I've had to reinstall many times, thus losing all my apps and settings. I now drag copy the PSF before making changes, to easily restore the OS if it breaks, without having to reinstall the OS. But takes time and disk space. Hoped rsync/snap2 would give me a better solution-- sounds like not.

Quote:
rsync will transfer files quickly by transferring only changed portions.

Sounds great. If "incremental" is the wrong word to use, my bad (although I have seen "incremental" used on more than one discussion of rsync). Are you saying rsync does or does not backup only changed files? Unclear. Also, you're saying snap2 is more "incremental" than rsync, or saves space better than rsync?

Quote:
if you are backing up a single huge file (Puppy save file) a new copy on disk will be necessary for every backup.

The puppy file system directory IS the puppy save file, is it not? Isn't backing up the file system equivalent to backing up the PSF? I'm no puppy expert. is reinstalling puppy the only way to benefit from an rsync "incremental" backup?

Quote:
/root' is the Puppy user 'home' directory and would be the default location to store documents and such. If you store them outside of puppy filesystem (on a /mnt/sdd1/ drive for example), that's OK too, and snap2 can always be used to back those up.

nice, but not trying to backup my docs. Trying to backup the OS. Again, does /root contain puppy desktop, configuration, and installed apps? Sorry for my ignorance.

Quote:
"Allow no hardlink reference" only allows a backup to proceed if there is no previous backup to use as a reference for the hardlinks.

then wouldn't that mean that leaving it checked would prevent backup attempts after the first one? Sounds like what you're saying.

Quote:
I think your best bet would be to use snap2 to backup the Puppy save file.

What's the advantage of that, over drag copy?

Quote:
You'd have to uncompress it first. You'd need an OS for that

Thats why I dual boot.

Sounds like snap2/rsync cannot be used for "incremental" backups of the OS Sad
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Fri 25 May 2012, 00:14    Post subject:  

1. Ok, I mean I don't use a backup program to recover a broken OS. As a sys admin, I find Debian suits me better than Puppy, and it never breaks. I promote Puppy for those whose computer use is less technical.

2. rsync alone is not incremental, although it can be used to back up only new/changed files, if that's what you mean. That will result in a single backup. snap2 is "more" incremental because it allows a series of backups.

3. "The puppy file system directory IS the puppy save file, is it not?" Yes and no. It is an "overlay" containing all the files that have changed or been added to the puppy RAM-based filesystem. Puppy RAM filesystem + save file = current state of filesystem. Reverting to a previous version of the save file will do what you want.

4. Lloyd: "Allow no hardlink reference" only allows a backup to proceed if there is no previous backup to use as a reference for the hardlinks."

Quote:
then wouldn't that mean that leaving it checked would prevent backup attempts after the first one? Sounds like what you're saying.


Golly, no. I'll put it another way: check it to *allow* a backup that will take up as much backup storage as the original files, in case a previous backup is not found. Uncheck it to *force* it to use hardlinks, meaning it will not do a backup at all if it can't locate a previous backup to use for the hardlinks. If unclear, just leave it checked and don't worry. It is only a convenience feature.

5. The advantage of snap2 for backing up a Puppy save file would be that only changed portions need be transferred, but it would take just as much backup storage as a regular copy. Also snap2 can automatically keep several older versions for you, automatically deleting the oldest. Maybe this is not enough advantage to make it worthwhile.

6. "Sounds like snap2/rsync cannot be used for "incremental" backups of the OS"
A series of save file backups can be automatically stored: yes. Incremental: No.

I finished fixing the bug in snap2 that caused backup exclusions not to work for the root directory ('/'), but some testing will necessary because the changes were not trivial.
Back to top
View user's profile Send private message 
johnywhy

Joined: 20 Aug 2011
Posts: 353

PostPosted: Fri 25 May 2012, 00:25    Post subject:  

Thanks for reply and info!

Anyone know of an "incremental" or differential backup method that can work with a single file? (such as the PSF, for example Smile. To backup just the changed parts of an individual file?

The PSF is already compressed, right?

Cheers!
Back to top
View user's profile Send private message 
don570


Joined: 10 Mar 2010
Posts: 3437
Location: Ontario

PostPosted: Sat 26 May 2012, 12:33    Post subject:  

Quote:
Anyone know of an "incremental" or differential backup method that can work with a single file?


There's the 'diff' command and then you use the 'patch' command

A bit complicated to use Crying or Very sad

For instance when a patch is applied to source code by a
programmer. This is the command I used to
apply a patch to mtpaint before compiling.

Code:
patch --input=/mnt/sdc1/tablet_test.patch --directory=/root/mtpaint3.34.56/src



http://linux.about.com/library/cmd/blcmdl1_diff.htm

http://linux.about.com/od/commands/l/blcmdl1_patch.htm

_________________________________________________

Last edited by don570 on Sat 26 May 2012, 12:55; edited 2 times in total
Back to top
View user's profile Send private message 
johnywhy

Joined: 20 Aug 2011
Posts: 353

PostPosted: Sat 26 May 2012, 12:48    Post subject:  

That rocks. Cannot wait to try it! Thanks!
Back to top
View user's profile Send private message 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sat 26 May 2012, 12:49    Post subject: rdiff-backup  

I think rdiff-backup would also work to keep one current copy of the puppy save file, and "binary diffs" that would allow recovery of a previous save file. From the program's features page:

"Space efficient: Suppose you have a large database file that changes a little bit every day. A normal incremental backup would keep saving copy after copy of this database, wasting a lot of space. rdiff-backup uses librsync, which implements the same efficient diffing algorithm that rsync uses. It works on binary files as well as text, so only a fraction of the data in your database would be saved in each incremental backup."

(As it says above, snap2 would store "copy after copy" of any large file - like the puppy save file - that changes.)

I found this here: http://www.nongnu.org/rdiff-backup/features.html

I don't know whether or not there is a Puppy pet, but I think rdiff-backup can be installed easily on Puppy.
Back to top
View user's profile Send private message 
peebee


Joined: 21 Sep 2008
Posts: 1546
Location: Worcestershire, UK

PostPosted: Sun 23 Sep 2012, 09:15    Post subject: Question  

A quick question if I may....

I've got Snap2 backing up from my working disk sda1 to my backup disk sdb1 at 19.00 each day.

All seems to be fine except that in my backup directory there is a directory for each day/backup which seems to contain just an empty /mnt directory - see screenie attached.

Can I safely delete all these daily directories??

Many thanks
Cheers
peebee
capture8222.png
 Description   
 Filesize   295.91 KB
 Viewed   445 Time(s)

capture8222.png


_________________
LxPup-14.10 = Puppy + LXDE
Back to top
View user's profile Send private message Visit poster's website 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sun 23 Sep 2012, 11:25    Post subject:  

With snap2, you can always safely delete any backup, unless it contains data you need. However, note that if you have empty daily.x directories, you probably have a misconfiguration. Check your latest backup (recent.1). If it is contains what you expect, you can probably conclude that your current configuration is OK.

In your screenshot I see a number of Backup_<date> directories. Those were created by a different backup program, right?
Back to top
View user's profile Send private message 
peebee


Joined: 21 Sep 2008
Posts: 1546
Location: Worcestershire, UK

PostPosted: Sun 23 Sep 2012, 12:07    Post subject:  

lstandish wrote:
In your screenshot I see a number of Backup_<date> directories. Those were created by a different backup program, right?


Hi lstandish

Thanks for the reply - no other backup program is in use - all those directories - 1 per day at 19.00 since I started using snap2 - are created by snap2 - they appear to be empty except for an empty /mnt directory in each.

The daily.x directories have the expected content.

Thanks
peebee

_________________
LxPup-14.10 = Puppy + LXDE
Back to top
View user's profile Send private message Visit poster's website 
lstandish


Joined: 06 Jun 2008
Posts: 126

PostPosted: Sun 23 Sep 2012, 12:23    Post subject:  

Something is fishy - snap2 doesn't create any directories called "Backup_<date>", not even temporary directories. Can you send me your /.snap2/default.set directory (compressed), maybe via PM?
Back to top
View user's profile Send private message 
peebee


Joined: 21 Sep 2008
Posts: 1546
Location: Worcestershire, UK

PostPosted: Sun 23 Sep 2012, 13:10    Post subject:  

lstandish wrote:
Something is fishy


OOOPS!

Profound apologies - my bad - looks like there was a pschedule task lurking from when I tried pbackup.

Really sorry to have bothered you!!!

Thanks
Peebee
capture7699.png
 Description   
 Filesize   20.72 KB
 Viewed   351 Time(s)

capture7699.png


_________________
LxPup-14.10 = Puppy + LXDE
Back to top
View user's profile Send private message Visit poster's website 
Pete22


Joined: 08 May 2009
Posts: 241
Location: Utah, USA

PostPosted: Mon 24 Sep 2012, 17:44    Post subject:  Does snap2 work on ext 4  

Hello lstandish,

I am going to bite the bullet and do what I should have been doing all along.
That is backing up. My system is Lhp 64 on ext.4
Does Snap2 work on ext.4 or should I reformat and go with ext.3?

Thanks so much for a nice program.


Pete
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 10 of 11 [158 Posts]   Goto page: Previous 1, 2, 3, ..., 8, 9, 10, 11 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1328s ][ Queries: 13 (0.0077s) ][ GZIP on ]