Speeding up the SnapMerge

[mavrothal]

You probably have an old version of the shutdown script.

Patriot's code looks like this and does work:

Actually this the the exact code I have and it works in the sense that shuts down the XO. The "problem" is that is doing it after a lazy umount of the save layer.
The "busybox mount -t $SAVEFS..." step reports "mounting /dev/mmcblk0p1 on /initrd/pup_rw failed: Device or resource busy"

Oh well, maybe is an XO specific issue

[jemimah]

It should mount the save layer read-only and then the lazy unmount. If the read-only step is successful, then the save file will be clean.

[mavrothal]

It should mount the save layer read-only

That's the exact step that fails with "Device or resource busy"

[jamesbond]

Tried s-17 (in XOpup).
I do not know if this a bug or a feature but when gnome-mplayer is open snapmergepuppy freezes in the initial "merge requested" dialog.
Even if mplayer is paused or stopped or even without a file loaded.
Resumes when mplayer closes.

There is nothing in messages or xerrs.log indicative of a problem other than a process can not be found to be stopped and then restarted, at the "kill -STOP ${allprocess[$i]}" and "kill -CONT $i" step respectively. But this message has nothing to do with gnome-mplayer running or not.

For comparison, running a flash movie in the browser pauses no problem in snapmerge (and only as small freeze in flash).

Hmm that's odd. How about this - open gnome-mplayer, and then open terminal, and run "snapmergepuppy" from terminal and tell me what you see. If everything works, the problem is elsewhere (snapmergepuppy is just the last chain of command that do the actual work, there are other scripts that work). If snapmergepuppy never finishes, (and you can press Ctrl-C to break out of it) - hmmm I need to think harder

[jamesbond]

I'm going to start a new save file and try to run in pupmode 13 for a few weeks and see how it goes.

I can have a break for a while

[mavrothal]

Hmm that's odd. How about this - open gnome-mplayer, and then open terminal, and run "snapmergepuppy" from terminal and tell me what you see. If everything works, the problem is elsewhere (snapmergepuppy is just the last chain of command that do the actual work, there are other scripts that work). If snapmergepuppy never finishes, (and you can press Ctrl-C to break out of it) - hmmm I need to think harder

You are actually right.
Is pup_event_frontend_d that does not allow merges by default when mplayer (and other apps) are open. However, save2flash does run and you end up with a hung warning screen.
Snapmergepuppy is fine
Get your rest

[jamesbond]

s18 - fix for those running bash version 4 (e.g. fatdog). For those not using bash4, there is no need to upgrade.

cheers!

[jamesbond]

s19 - use --force option for rsync to make sure that it we replace a directory with a symlink at the top layer, this will be rsync-ed down to the save layer.

I'm cleaning up and removing all the old versions.

[jamesbond]

Been eating my own dogfood for a week now, running this on 1GB pupsave file on top of 16GB ext3-formatted flash. I bumped up the size of the warning from 32MB in the script to 128MB as I don't notice a delay during merge.

So far looks okay, no data loss, no odd behaviour, etc.

cheers!

[jamesbond]

The story three weeks later - so far so good. The save is almost always instantaneous - for 200MB worth of data (this is on a 4GB machine - so it appears that the kernel manage to cache most of it - different machines may fare differently). The delay however is apparent when we do "sync".

[InTheDogHouse]

I have a 700MB save file in Puppy Precise 5.7.1, with the normal collection of small files, plus a 200 MB TrueCrypt container file (and yes, I am following replacement options for the now possibly insecure TrueCrypt).

Downloaded and installed snapmerge-19.pet, to see if it would speed up shutting down (and saving) Puppy.

Some improvement in save time, but not dramatic.

Googled rsync, discovered that when copying to/from a remote connection, rsync copies only the changed portions of files, dramatically speeding the copy process. BUT, when copying locally (as in saving Puppy session from ram to my save file) rsync defaults to copying everything, not just changed portions of files. So slightest change in my 200 MB TrueCrypt volume (file) causes a 200 MB copy!

So I modified snapmerge's rsync to have the additional parameter '--no-whole-file'. This tells rsync to only copy changed portions of files, always, on local system copies, as well as being the default on remote copies.

The result of '--no-whole-file' is amazing, cutting my save time in half, or even less if file changes during my session have been minimal!

My question: Is this '--no-whole-file' modification a great idea, or will there be unanticipated consequences that will make me wish I had never made this change???

[amigo]

I'd be concerned about data corruption if this snap-thing crashes or gets interrupted in the middle of it's job.