Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 19 Dec 2014, 03:24
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Unsorted
clean_whiteouts_daemon: A Daemon to Clean Whiteout File
Moderators: deshlab, Flash, GuestToo, Ian, JohnMurga, Lobster
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [10 Posts]  
Author Message
Dougal


Joined: 19 Oct 2005
Posts: 2505
Location: Hell more grotesque than any medieval woodcut

PostPosted: Tue 03 Mar 2009, 18:02    Post subject:  clean_whiteouts_daemon: A Daemon to Clean Whiteout File
Subject description: So you don't get "no space left on device" messages
 

I noticed that yet another person was hurt by the whiteout files resulting from Opera's cache activity, so I figured: "someone should add the cleanup once and for all to the Puppy shutdown script".
But then I thought, "But why do we need to reboot for them to be cleaned? This isn't Windows, right?"

So I wrote this daemon, to take care of the whiteout files.

What it does is sleep for some time (default is 30 minutes), then wake up and go cleaning. Then go to sleep again. And so on.
It doesn't just delete whatever it finds -- it makes sure they don't cover files in any union layer.

By default it checks all the filesystem for whiteout files, but if you want it to only look at some directories (e.g. /root), you have a config option.
There's also an option to have /tmp cleaned (since a lot of whiteout files can be created there when compiling), but I think Barry reinstated making /tmp a tmpfs, so it might not be necessary.
You can make it verbose and it will list every file it deletes in the log file (/var/log/clean_whiteouts.log).

There are some more config options, all explained in the config file (/etc/clean_whiteouts.conf).

In case anybody is worried about performance (i.e. it bloating Puppy), I don't think that should be much of a problem, as the script renices itself so it runs at a low priority and should not steal the cpu when other apps want it.

It comes with a startup file in /etc/init.d, so it will be started at bootup and stopped at shutdown (the dotpet will also start it when it installs).

It should also help make shutdown faster for people running off flash drives (PUPMODE 3/7/13), as snapmergepuppy will have less files to handle...

I haven't tested it much, but similar code was in the shutdown script for 2.14R... maybe people should test it a bit (keeping an eye on the log file) and report here if it works ok or not.

(In case anyone wonders why the script name is strangely shorter that it's title, that's because killall is stupid and can't handle long names...)

_________________
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind

Last edited by Dougal on Sun 12 Apr 2009, 05:32; edited 1 time in total
Back to top
View user's profile Send private message 
WhoDo


Joined: 11 Jul 2006
Posts: 4441
Location: Lake Macquarie NSW Australia

PostPosted: Tue 03 Mar 2009, 18:48    Post subject: Re: clean_whiteouts_daemon: A Daemon to Clean Whiteout File
Subject description: So you don't get "no space left on device" messages
 

Dougal wrote:
So I wrote this daemon, to take care of the whiteout files.

Looks like a must-have feature for Puppy, Dougal. Cool I'll watch the testing with interest and if there aren't any major stumbling blocks then I'll add it to Puppy-4.2RC2

Thanks, mate.

_________________
Actions speak louder than words ... and they usually work when words don't!
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com
Back to top
View user's profile Send private message 
MU


Joined: 24 Aug 2005
Posts: 13644
Location: Karlsruhe, Germany

PostPosted: Tue 03 Mar 2009, 18:51    Post subject:  

I think this will just be needed in old Puppys up to 4.0 alpha.

They used the unionfs.ko module as unionfs driver to set up the layered filesystem.

Newer versions instead use aufs.ko, an alternative, better unionfs driver.
That one should no longer have that issue.

Or just for clarification, where there reports for the new versions of Puppy, too? (4.1 and newer)
Mark

_________________
my recommended links
Back to top
View user's profile Send private message Visit poster's website 
WhoDo


Joined: 11 Jul 2006
Posts: 4441
Location: Lake Macquarie NSW Australia

PostPosted: Tue 03 Mar 2009, 19:11    Post subject:  

MU wrote:
I think this will just be needed in old Puppys up to 4.0 alpha.

They used the unionfs.ko module as unionfs driver to set up the layered filesystem.

Newer versions instead use aufs.ko, an alternative, better unionfs driver.
That one should no longer have that issue.

Or just for clarification, where there reports for the new versions of Puppy, too? (4.1 and newer)

I'm not aware of any reports of the problem where aufs.ko is used by default, Mark. What about the option to use unionfs.ko in later Puppy's with a boot parameter? Also doesn't the boot revert to unionfs.ko if aufs.ko is not available for some reason? In these cases, wouldn't Dougal's daemon provide additional fault protection? Confused Maybe we could include it and have it enabled manually rather than automatically, so giving users the option. Idea

These are just my thoughts on the subject. If you guys tell me it isn't needed at all for 4.2 with aufs.ko then I'll just forget about it. Smile

_________________
Actions speak louder than words ... and they usually work when words don't!
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com
Back to top
View user's profile Send private message 
Dougal


Joined: 19 Oct 2005
Posts: 2505
Location: Hell more grotesque than any medieval woodcut

PostPosted: Wed 04 Mar 2009, 07:25    Post subject:  

MU wrote:
I think this will just be needed in old Puppys up to 4.0 alpha.

They used the unionfs.ko module as unionfs driver to set up the layered filesystem.

Newer versions instead use aufs.ko, an alternative, better unionfs driver.
That one should no longer have that issue.

I think the newer versions use aufs, but you have a boot option (layerfs) to use unionfs.

I myself have been using unionfs, since I compiled my own kernel and the unionfs patch adds the module to the kernel tree, whereas the aufs patch just makes some changes to the kernel and expect you to compile aufs.ko after you've booted the new kernel...

_________________
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind
Back to top
View user's profile Send private message 
MU


Joined: 24 Aug 2005
Posts: 13644
Location: Karlsruhe, Germany

PostPosted: Wed 04 Mar 2009, 13:25    Post subject:  

I see, so your utility really is important.
The whiteout files can completely destroy a savefile, this drove me crazy in Muppy, when we did not use aufs.ko yet.

Mark

_________________
my recommended links
Back to top
View user's profile Send private message Visit poster's website 
Dougal


Joined: 19 Oct 2005
Posts: 2505
Location: Hell more grotesque than any medieval woodcut

PostPosted: Thu 05 Mar 2009, 07:47    Post subject:  

Before anyone goes adding this to any kind of release, let me first make a little improvements to it, so that it will only run when using unionfs -- no need for sufs or a full install...
_________________
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind
Back to top
View user's profile Send private message 
WhoDo


Joined: 11 Jul 2006
Posts: 4441
Location: Lake Macquarie NSW Australia

PostPosted: Thu 05 Mar 2009, 18:27    Post subject:  

Dougal wrote:
Before anyone goes adding this to any kind of release, let me first make a little improvements to it, so that it will only run when using unionfs -- no need for sufs or a full install...

Done. Cool

_________________
Actions speak louder than words ... and they usually work when words don't!
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com
Back to top
View user's profile Send private message 
big_bass

Joined: 13 Aug 2007
Posts: 1747

PostPosted: Sat 07 Mar 2009, 20:58    Post subject:  

Dougal

I am testing your pet I am doing an experimental USB build of
woof and I need all the extra space I can get

I am multi booting 5 OS's so I ran out of room on my hard drive
for the required 10 GB of space needed
so I do need to test your cleaner


a side note:
and just something I came across looking over woof
the new woof is handing it like this
in this script
/root/woof-alpha2/boot/initrd-tree0/init


there is this comment
#w003 /usr/sbin/snapmergepuppy now saves whiteout files to save-layer, change 'ro' to 'ro+wh'.

can that be of use to you ?



big_bass
Back to top
View user's profile Send private message 
disciple

Joined: 20 May 2006
Posts: 6464
Location: Auckland, New Zealand

PostPosted: Mon 09 Mar 2009, 02:36    Post subject:  

Quote:
only run when using unionfs

Even if you don't have the unionfs problem, the large number of whiteout files that can accumulate can be a little annoying Smile

_________________
DEATH TO SPREADSHEETS
- - -
Classic Puppy quotes
- - -
Beware the demented serfers!
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [10 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Unsorted
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 2.3146s ][ Queries: 11 (2.2458s) ][ GZIP on ]