aufs for Puppy

Under development: PCMCIA, wireless, etc.
Message
Author
User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

aufs for Puppy

#1 Post by Nathan F »

The files needed to get aufs working in Puppy-2.14 are here. This tarball contains a patched initrd.gz with the aufs module and a pet package that contains the aufs utilities. Instructions:
  • 1) untar this archive
    2) do a normal frugal install (should also work on a usb stick but I don't have one to sacrifice right now)
    3) replace the initrd.gz that Puppy will install onto the hard drive with the one in this archive.
    4) boot it up
    5) (optional) install the pet package with the aufs utilities
The last step should only be necessary if you want to test aufs for things like swapping out branches on the fly, otherwise it serves no useful purpose. Of course the ability to do that is the motivating factor for why I compiled it at all. I still haven't tested whether or not it will do the trick or not, but intend to do so very soon. In any event, I think it would be good to move Puppy towards using aufs anyway, as unionfs is know to be buggy and aufs implements new features.

This was surprisingly easy to get working, I wish I had tried sooner.

Nathan
Bring on the locusts ...

User avatar
Lobster
Official Crustacean
Posts: 15522
Joined: Wed 04 May 2005, 06:06
Location: Paradox Realm
Contact:

#2 Post by Lobster »

I am an oaf when it comes to aufs

are they related to awks?

:oops: wot is it?
Puppy Raspup 8.2Final 8)
Puppy Links Page http://www.smokey01.com/bruceb/puppy.html :D

User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

#3 Post by Nathan F »

Aufs is a total rewrite and new implementation of unionfs, with supposedly much fewer bugs and enhanced features. There is quite a discussion going on elsewhere about the ability to add or remove union branches from the main filesystem, and many were hoping that aufs would make this possible. This would enable us to add or remove squashfile extension while the system was running, which greatly improves the flexibility and modularity of Puppy. Even if it didn't work, it would seem that aufs is going to provide better performance and reliability than unionfs ever has.

This is provided for testing by developers, although I don't think it will harm your system in any way. But it's pretty useless for end users right now, since it just does the same thing that unionfs is already doing in Puppy. But it may enable a lot of other cool things in the future.

Nathan
Bring on the locusts ...

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#4 Post by sunburnt »

Lobster; It's the newer replacement for UnionFS that joins these 2 file's: pup_xxx.3fs & pup_xxx.sfs.
As Nathan said, UnionFS is buggy & not as capable as AUFS (Another Union File System).

Darn it! Nathan beat me to it!

Nathan; I'll check it out tonight & let you know...

User avatar
Lobster
Official Crustacean
Posts: 15522
Joined: Wed 04 May 2005, 06:06
Location: Paradox Realm
Contact:

#5 Post by Lobster »

thanks guys 8)
Puppy Raspup 8.2Final 8)
Puppy Links Page http://www.smokey01.com/bruceb/puppy.html :D

User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

#6 Post by Nathan F »

sunburnt (& anyone else who may have already downloaded) -

I tried it out and it is possible to add and remove branches on the fly, in fact it works beautifully now. But the included unionctl script does not work, and it requires two small utilities that are not in Puppy by default. I re-uploaded the taqrball without the unionctl script, and with the utilities rev and comm, so if you have already installed the aufs utilities package please remove and then reinstall with the new one.

To add a branch:

Code: Select all

mount-FULL -o remount,append:/initrd/pup_ro5 /
To remove a branch:

Code: Select all

mount-FULL -o remount,del:/initrd/pup_ro5 /
Replacing /initrd/pup_ro5 with whatever directory you want to add or remove. Use mount-FULL rather than just mount, as mount is a shell script in Puppy that calls busybox mount. The mount-FULL binary is the proper mount program with all features.

I added and removed several branches without any problems, ran losetup to check for the next available loop without errors, and rebooted no problem. I have yet to see if it will cause any stability issues (ie the system eventually slows down like it did with unionfs after adding a branch) but after these tests I'm both impressed and highly optimistic. This means I can get back to work soon on my squash program and port all of the code from Puppy-1.xx to Puppy-2.xx, and we should be able to add and remove squashfile extensions at will. Just as we were hoping :D

Any testing anyone else can provide would be highly appreciated here, to confirm my results.

Nathan
Bring on the locusts ...

raffy
Posts: 4798
Joined: Wed 25 May 2005, 12:20
Location: Manila

Good news

#7 Post by raffy »

Very very interesting - sunburnt must be smiling - no, laughing! :D
Puppy user since Oct 2004. Want FreeOffice? [url=http://puppylinux.info/topic/freeoffice-2012-sfs]Get the sfs (English only)[/url].

kirk
Posts: 1553
Joined: Fri 11 Nov 2005, 19:04
Location: florida

#8 Post by kirk »

Nathan,
I tried it out and it is possible to add and remove branches on the fly, in fact it works beautifully now.
Wow, that sounds great. Tried to down load your package, but couldn't log in to you ftp server. Tried anonymous, no luck.

Thanks,

Kirk

User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

#9 Post by Nathan F »

I'll post it elsewhere with an http link.

Nathan
Bring on the locusts ...

kirk
Posts: 1553
Joined: Fri 11 Nov 2005, 19:04
Location: florida

#10 Post by kirk »

Link working now. Thanks Nathan.

kirk
Posts: 1553
Joined: Fri 11 Nov 2005, 19:04
Location: florida

#11 Post by kirk »

Works for me!

And it seems to be slightly faster too.


I tried to remove a branch, but I got this:

sh-3.00# mount-FULL -o remount,del:/initrd/pup_ro4 /
mount: invalid option -- i
sh-3.00#


The -i seems strange, the pup_ro4 on my rig is the kernel source, so it shouldn't have had anything in use. I haven't read all the Docs yet, I'll work on it some more.

Should I have ran any of these scripts/binarys that where put in /sbin after the install?

aufind.sh
aulchown
auplink
comm
mount.aufs
rev
umount.aufs

User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

#12 Post by Nathan F »

Subnurnt has been experiencing this same error, and frankly I don't know where it's coming from. I'm not getting it myself, but I might be doing something slightly different. It's on my to do list today to check more into this.

For the record, Kirk, can you give me the exact commands you are using to mount a squashfile and add the branch, and then top remove it? I have a certain suspicion I want to check out.

I don't know what every script in there does (I still need to look over the docs more myself) but I can explain a couple of them.

When adding a branch using "mount-FULL -o remount" /etc/mtab does not get updated, so typing mount will not show the correct branches in your aufs union. The mount.aufs script is supposed to fix this, but I haven't gotten it to work yet. However, if you read /proc/mounts it will show the correct information so mtab doesn't really bother me too much right now.

The comm command is a *nix utility that is missing from Puppy, which is required by mount.aufs, so I put it in. An online man page for it is here, if you want to know what it does.

Similarly, the rev binary is another *nix utility that is not in the standard Puppy. It was needed by the unionctl script that is designed to emulate the command by the same name in unionfs. I could not get unionctl to work either, so it can probably be removed. The rev command basically reverses the letters in a string.

The auplink program has something to do with the symbolic links in an aufs branch, but I need to read more about it. The rest I don't know enough about (yet) to comment on.

Nathan
Bring on the locusts ...

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#13 Post by sunburnt »

The mount command showed the correct aufs line for me.
The added branch was the last one of course, as it was added with "append".

Nathan; What Puppy-2 version are you using?

If it just has trouble with certain ones, that could be acceptable.
Although it would really be nice to know what the actual problem is.

User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

#14 Post by Nathan F »

I think I found it, but not sure. It seems the mount.aufs and umount.aufs scripts are actually used even if not called directly. When I looked in them they of course called mount rather than umount. I've re-uploaded the tarball after adding the full umount binary (rather than the busybox applet) as umount-FULL, and changing references in the aufs scripts to point to mount-FULL and umount-FULL.

The -i option was being passed to umount through the umount.aufs script, which is where I believe the errors were coming from (although I could possible be wrong).

I've also uploaded just the utils package, without the initrd.gz, so you can just replace that part.
aufs utils package

Let me know if this fixes the errors you have been experiencing, or makes any change whatsoever. I think this just might do it.

Nathan
Bring on the locusts ...

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#15 Post by sunburnt »

KILLER !!! I'll test as soon as I get to my machine... let you know.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#16 Post by sunburnt »

I'm wondering what the umount-FULL is for, as the aufs commands are all:

mount-FULL -o remount,append:/initrd/pup_ro3 /
mount-FULL -o remount,del:/initrd/pup_ro3 /

It didn't work when I tried: umount-FULL -o remount,del:/initrd/pup_ro3 /
It froze same as before without the "-- i" error.

I don't think it's right, so I didn't try: umount-FULL /initrd/pup_ro3


Nathan; Are we suppost to be using a different un-mounting command?

Are you using GraphPup for testing this... or what? Maybe it's different somehow.

kirk
Posts: 1553
Joined: Fri 11 Nov 2005, 19:04
Location: florida

#17 Post by kirk »

For the record, Kirk, can you give me the exact commands you are using to mount a squashfile and add the branch, and then top remove it? I have a certain suspicion I want to check out.
I haven't tried to add a branch yet. I was trying to remove a branch that init had added. My first post has the exact (only) command I was trying to use. Could init be adding branches in some different fashon?

Couldn't download your new package, I'll try again later tonight. Don't have time to test right now anyway. I'll also try adding a branch and then try to remove it.

Thanks again.

User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

#18 Post by Nathan F »

I'm wondering what the umount-FULL is for
If you look in the script umount.aufs that comes with aufs, it calls umount. Since umount is a link to busybox in Puppy and cannot accept the -i option I replaced it with the full binary, and changed the umount.aufs script to place the correct call. The commands add or remove branches are exactly the same as I mentioned before, and the aufs kernel module handles the rest.
Bring on the locusts ...

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#19 Post by sunburnt »

Sorry, I caught that from your other post, but thought it had another purpose.
It solved the error: Invalid argument -- i
But the crash after that is apparently something different.

If it works for you, need to know why that Puppy version works & not others.

Terry

User avatar
Nathan F
Posts: 1764
Joined: Wed 08 Jun 2005, 14:45
Location: Wadsworth, OH (occasionally home)
Contact:

#20 Post by Nathan F »

I'm using 2.14 with the patched initrd.gz and aufs_utils package I posted.

Nathan
Bring on the locusts ...

Post Reply