loadAndUnloadBranchDirs.sh - to the aufs stack

Under development: PCMCIA, wireless, etc.
Message
Author
User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#21 Post by mikeb »

pupmode=13 has only 1 writable layer. "snapmergepuppy" script writes to the save-layer by accessing the directories directly, outside aufs
erm pup_ro1 is a aufs layer...yes snapmerge writes directly but inherently you are writing to a aufs layer active in the union. Which also supports the more than one rw layer thought. Nothing goes on outside of the union/aufs.

Perhaps we are at cross definitions here.

mike

User avatar
mavrothal
Posts: 3096
Joined: Mon 24 Aug 2009, 18:23

#22 Post by mavrothal »

gyro,
looking at your recent delving into aufs, I was wondering what do you think about an aufs-based full install of puppy i.e. a full install that would allow use of SFSs without the need of chroot, symlinks or anything else.
I would think the /sbin/init that is currently used in full installs to do a filysystem check (if needed) before passing it on the busybox init, could be modified to setup a layered system with a single rw branch, the root partition in full installs. Then SFSs could then be added on the top of the bottom rw branch so they could modify core functions as needed.
Would this be possible with aufs or it would then need an additional top rw branch to introduce whiteouts as needed?
If another rw top-layer is needed, then unmounting the SFSs should also check and remove the possible top-layer whiteouts for the specific branch. At the end removing all layers but the bottom and top rw branches should merge down the top branch and delete it (or maybe some other approach leading to a similar end point, i.e. one writable branch what no other ro branches are present).
What do you (or others) think?
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#23 Post by mikeb »

well that would seem like the main system is then behaving like the initrd ....thing is with any complete union of layers (ie '/') it has to be unioned to somewhere and then you chroot into it to use it... nature of the beast.

Partial unions are more flexible..ie a particular subfolder...indeed there is not problem doing that with a full install. eg union to say /usr/local .. a knoppix/puppy 1 like approach. Indeed those seems where this thread started off and applications from that. Not helpful of course with sfs made to mount on '/' and who would like to make a third set of puppy packages... I would assume not.

I pulled in the idea of adding a shared config along the partial union approach as it seem there may be a possible merge of ideas.

No answers just trying to summarise my view of this so far.
One of those 'you can almost taste it' moments...

If a layer other than the top can be read/write then such a layer could be manipulated in the way of mount and copy into it an sfs for example aka how I use a sfs save but after the union is made and running. That same layer could equally be cleared out and replenished. Like a temp/disposable software or settings layer.
How tolerant is aufs of doing that to it?
No help for the full install idea but aimed at union variations/shared storage.

mike

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#24 Post by gyro »

@All folks:
Please read the man page for aufs http://aufs.sourceforge.net/aufs2/man.html
In particular the section called "User's Direct Branch Access (UDBA)" to understand the difficulty aufs has with direct access to a branch directory like "snapmergepuppy" does.
Also the section called "Policies to Select One among Multiple Writable Branches" to understand that aufs does support multiple writable branches, but the difficulties are in getting it to write the appropriate stuff to the appropriate branch.

gyro

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#25 Post by mikeb »

yes indeed..it does not like it...spend time playing with pup_rw and see how long before the system chokes.

The 'safe' way to do it is to unload , then write, then re insert.... a bit beyond the puppy way of things.
That in itself is not exactly useful for a dynamic settings storage though.

Anyway I guess we are just toying with ideas and possibilities.

For full..what is the objection to chroot... seemed pretty neat to me though might be a problem if interacting with other apps?
I also was not sure why aufs could not be used for it.

mike

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#26 Post by gyro »

@mavrothal,
This is the closest I've come to a full install in a layered system,http://www.murga-linux.com/puppy/viewtopic.php?t=99376.
The next step would be create a system with the rw layer being in tmpfs, and the whole system in a single directory as a ro branch. Sort of a pupmode=13, with the whole system in the save-layer, and no other layers.
Then try to pupmode=12 that, so the whole system is in the save-layer which is an rw branch.
Then redo all that but with the directory being the root of a partition.

So it would probably be easier to start with a frugal install and see how close to a full install you can get.

But in the end, is it really a full install?
While the contents of the partition will look like a full install, the real root of your system is the unionfs, not the root directory of the partition. You may have to boot into a filesystem that is not the partition, setup the unionfs that includes the partition as a branch, and chroot into the unionfs, as the 'init' script does in a frugal install.

I think that any sfs files would have to be appended as ro branches below the rw branch.

Another thing to watch with aufs is that, in all my testing, the branch directories have been beside each other, one branch is never the obvious parent or sub-directory of another branch. So, to add a partition as a branch you probably can't have any other branches on the same partition. (Hmmm... that might not be a problem.)

gyro

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#27 Post by mikeb »

Well the closest frugal/full setup I added involved a save image file in which a full install was made...the initrd simply mounted it then chrooted into it.... I kept it as a save option after trying it in puppy 2.12.

So no union , behaves like a full and can exist as one file on fat32 or ntfs.

But the interest here is to add applications and settings in a non permanent way really and some form of union is all I can think of.

mike

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#28 Post by gyro »

mikeb wrote:The 'safe' way to do it is to unload , then write, then re insert.... a bit beyond the puppy way of things.
That in itself is not exactly useful for a dynamic settings storage though.
But the fact that pupmode=13 works for some folks, means that "snamergepuppy" makes a passable fist of writing directly and then telling aufs that things have changed.
Edit:
So, to make a "dynamic settings storeage" layer it's contents would probably have to be filled by some "snapmergepuppy" like script that moved the relevant files.
mikeb wrote:Anyway I guess we are just toying with ideas and possibilities.
Yes we are.

gyro
Last edited by gyro on Tue 02 Jun 2015, 16:14, edited 1 time in total.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#29 Post by gyro »

mikeb wrote:But the interest here is to add applications and settings in a non permanent way really and some form of union is all I can think of.
Yes, I can not see a way to use sfs files in the way they are used in a frugal install without some sort of unionfs.
gyro

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#30 Post by mikeb »

lol...well snapmerge has room for improvement but i felt it was a hybrid cludge that seemed obsolete for anything usb2 or newer..or load all to ram instead as there is more of that around too...but i digress.

You also get quirks like
http://www.murga-linux.com/puppy/viewtopic.php?t=99549

selective snapping for a shared profile for those who can never have enough pups installed?

mike

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#31 Post by gyro »

@mikeb,
I just took my own advice and had another look at the aufs man page in the "Policies to Select One among Multiple Writable Branches" section, where it says "Selects the highest writable branch where the parent dir exists." So, if there were 2 rw branches, and all the parent dir's for all required settings files were defined in the lower rw branch and not in the top rw branch, then maybe the appropriate files will be written to the lower rw branch. If aufs actually works according to it's doco.

Do you have any suggestions as to an application and it's setting file that could be used as a test?

gyro

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#32 Post by mikeb »

hmm well a popular one would be something like firefox or seamonkey as profile sharing is common.
So you can effectively toggle which rw layer is used ... sounds neat if possible.

mike

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#33 Post by gyro »

It seems to work.
1) created a directory called "config" in my frugal install directory.
2) created "/root/.mozilla" directory within it.
3) removed "/root/.mozilla" from "tahrsave"
4) appended "config" as an rw branch at the bottom of the aufs stack.
5) started firefox
6) created a new file "/root/my-applications/lib/x.txt"
7) edited /etc/rc.d/rc.shutdown"

firefox created all it's ".mozilla" stuff in the "config" branch.
both "x.txt" and "rc.shutdown" were written to the "tahrsave" layer.

Yes, it seems a bit odd to append an rw layer at the bottom, but when I inserted it just below the "tahrsave" layer, the new file "x.txt" was written to it as well as the ".mozilla" stuff.

So:
1) aufs does support multiple rw layers.
2) the contents of an rw layer can be controlled by ensuring the parent directory is already present in the layer.
3) if the rw layer is above the ro layers, it could get unwanted new files, so append it to the bottom. This could still be useful for a common config layer, provided the appropriate config directory/file is not present in the ro layers.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#34 Post by gyro »

But for shared configs, why not simply create a directory somewhere under /mnt/home and symlink it to the appropriate directory in /root, in each puppy that needs to share it.
gyro

User avatar
mikeb
Posts: 11297
Joined: Thu 23 Nov 2006, 13:56

#35 Post by mikeb »

Nice one for testing.... I suppose it depends on how many profiles are shared as to the gain. (or the whole of /root....)

I believe there was also an interest in avoiding the mess that ensues when not only configs but several apps are also shared and stored outside the puppy save system.

We also have the load sfs to full install thoughts too..

In my world I just use multiple sfs and be done with it and share them when they can be... but many like the more complicated life and use the aforementioned schema to avoid overstuffing those flaky save files.

Or use a save folder if you don;t want to worry about space.

Why not use a frugal layered system as it was intended instead of peeing around the issue eh :D

mike

Post Reply