(old)Puppy Linux Discussion Forum

I noticed that yet another person was hurt by the whiteout files resulting from Opera's cache activity, so I figured: "someone should add the cleanup once and for all to the Puppy shutdown script".
But then I thought, "But why do we need to reboot for them to be cleaned? This isn't Windows, right?"

So I wrote this daemon, to take care of the whiteout files.

What it does is sleep for some time (default is 30 minutes), then wake up and go cleaning. Then go to sleep again. And so on.
It doesn't just delete whatever it finds -- it makes sure they don't cover files in any union layer.

By default it checks all the filesystem for whiteout files, but if you want it to only look at some directories (e.g. /root), you have a config option.
There's also an option to have /tmp cleaned (since a lot of whiteout files can be created there when compiling), but I think Barry reinstated making /tmp a tmpfs, so it might not be necessary.
You can make it verbose and it will list every file it deletes in the log file (/var/log/clean_whiteouts.log).

There are some more config options, all explained in the config file (/etc/clean_whiteouts.conf).

In case anybody is worried about performance (i.e. it bloating Puppy), I don't think that should be much of a problem, as the script renices itself so it runs at a low priority and should not steal the cpu when other apps want it.

It comes with a startup file in /etc/init.d, so it will be started at bootup and stopped at shutdown (the dotpet will also start it when it installs).

It should also help make shutdown faster for people running off flash drives (PUPMODE 3/7/13), as snapmergepuppy will have less files to handle...

I haven't tested it much, but similar code was in the shutdown script for 2.14R... maybe people should test it a bit (keeping an eye on the log file) and report here if it works ok or not.

(In case anyone wonders why the script name is strangely shorter that it's title, that's because killall is stupid and can't handle long names...)

Dougal wrote:So I wrote this daemon, to take care of the whiteout files.

Looks like a must-have feature for Puppy, Dougal. I'll watch the testing with interest and if there aren't any major stumbling blocks then I'll add it to Puppy-4.2RC2

Thanks, mate.

I think this will just be needed in old Puppys up to 4.0 alpha.

They used the unionfs.ko module as unionfs driver to set up the layered filesystem.

Newer versions instead use aufs.ko, an alternative, better unionfs driver.
That one should no longer have that issue.

Or just for clarification, where there reports for the new versions of Puppy, too? (4.1 and newer)
Mark

MU wrote:I think this will just be needed in old Puppys up to 4.0 alpha.

They used the unionfs.ko module as unionfs driver to set up the layered filesystem.

Newer versions instead use aufs.ko, an alternative, better unionfs driver.
That one should no longer have that issue.

Or just for clarification, where there reports for the new versions of Puppy, too? (4.1 and newer)

I'm not aware of any reports of the problem where aufs.ko is used by default, Mark. What about the option to use unionfs.ko in later Puppy's with a boot parameter? Also doesn't the boot revert to unionfs.ko if aufs.ko is not available for some reason? In these cases, wouldn't Dougal's daemon provide additional fault protection? Maybe we could include it and have it enabled manually rather than automatically, so giving users the option.

These are just my thoughts on the subject. If you guys tell me it isn't needed at all for 4.2 with aufs.ko then I'll just forget about it.

MU wrote:I think this will just be needed in old Puppys up to 4.0 alpha.

They used the unionfs.ko module as unionfs driver to set up the layered filesystem.

Newer versions instead use aufs.ko, an alternative, better unionfs driver.
That one should no longer have that issue.

I think the newer versions use aufs, but you have a boot option (layerfs) to use unionfs.

I myself have been using unionfs, since I compiled my own kernel and the unionfs patch adds the module to the kernel tree, whereas the aufs patch just makes some changes to the kernel and expect you to compile aufs.ko after you've booted the new kernel...

I see, so your utility really is important.
The whiteout files can completely destroy a savefile, this drove me crazy in Muppy, when we did not use aufs.ko yet.

Mark

Before anyone goes adding this to any kind of release, let me first make a little improvements to it, so that it will only run when using unionfs -- no need for sufs or a full install...

Dougal wrote:Before anyone goes adding this to any kind of release, let me first make a little improvements to it, so that it will only run when using unionfs -- no need for sufs or a full install...

Done.

Dougal

I am testing your pet I am doing an experimental USB build of
woof and I need all the extra space I can get

I am multi booting 5 OS's so I ran out of room on my hard drive
for the required 10 GB of space needed
so I do need to test your cleaner


a side note:
and just something I came across looking over woof
the new woof is handing it like this
in this script
/root/woof-alpha2/boot/initrd-tree0/init


there is this comment
#w003 /usr/sbin/snapmergepuppy now saves whiteout files to save-layer, change 'ro' to 'ro+wh'.

can that be of use to you ?



big_bass

only run when using unionfs

Even if you don't have the unionfs problem, the large number of whiteout files that can accumulate can be a little annoying