Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 19 Dec 2014, 12:52
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
using SFS with full installs
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [22 Posts]   Goto page: 1, 2 Next
Author Message
mikeb


Joined: 23 Nov 2006
Posts: 9022

PostPosted: Sat 21 Dec 2013, 12:34    Post subject:  using SFS with full installs  

After recently being introduced by amigo to a method of running an application in a layered fashion without touching the base file system my mind wandered into potential uses.

One then popped in and that was as a way of using sfs files with a full install which is a common request and usually involves extracting, or find a pet , or elaborate methods of linking, or other such methods.

Amigo's chrooted system seems to fit the bill nicely and a quick test in a virtual machine looks promising...I used an Xchat sfs and it ran normally.

Basic overview...
make folders... 'union', 'write' and 'app' somewhere.
mount the sfs to app.
make a fuse unionfs to 'union' using 'write', 'app' and '/'
fakechroot or chroot union /path/to/app
done.
In effect a unionfs is made using the full install as the bottom read only layer, the mounted sfs is next and a read/write layer on top.

It seems simple , does not care where the layers are located which is handy if confusing to me, and indeed solves a problem.

Don't expect replies...just posting for posterity and to keep notes.

mike
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5043
Location: Arizona, U.S.A.

PostPosted: Sat 21 Dec 2013, 13:32    Post subject:  

Mike; I think the newer Puppies will union an SFS file over a Full install.

Back in the early days I wondered why this didn`t work. Probably unionfs...
But I use Frugal installs so this wasn`t a priority.
Thinking about it, what`s the difference between unioning a mounted SFS file or /.?
In both cases you union on another dir. and chroot into it.
.
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9022

PostPosted: Sat 21 Dec 2013, 13:45    Post subject:  

Quote:
Mike; I think the newer Puppies will union an SFS file over a Full install.

hmm maybe so...but I failed with aufs a while ago. Would be nice to know one way or the other though seems odd if the question still gets asked.

What I could do was union to a folder of a full install ..eg /usr..

Well I have a full install of lucid running at the moment..I will test that...if anyone has anything newer would be of interest.

mike
Back to top
View user's profile Send private message 
L18L

Joined: 19 Jun 2010
Posts: 2610
Location: www.eussenheim.de/

PostPosted: Sat 21 Dec 2013, 13:57    Post subject: using SFS with full installs
Subject description: sfs_load on Quirky6.0
 

mikeb wrote:
...if anyone has anything newer would be of interest.

Quirky 6.0
log of loading a devx sfs....

# make
bash: /usr/bin/make: Datei oder Verzeichnis nicht gefunden
# LANGUAGE=en sfs_load
sfs_load: Your Puppy is running in a mode without extra SFS support. But this program can install SFSes using pseudo-PET. It is not exactly the same as loading SFS but works well in most cases.
WARNING: Using this under 'PUPMODE=2' is EXPERIMENTAL.

(gtkdialog4:12646): GLib-GObject-WARNING **: gsignal.c:3081: signal name `depressed' is invalid for instance `0x8507878'
/usr/sbin/sfs_load: line 774: 12586 Terminated $GTKDIALOG -p DIALOG -c > /dev/null
cp: cannot create symbolic link `/usr/include/gphoto2/gphoto2/gphoto2': No such file or directory


gtk-update-icon-cache: Cache file created successfully.
Generating /root/.icewm/menu...
Generating /root/.jwmrc...
/usr/sbin/sfs_load: line 369: 12958 Terminated $GTKDIALOG -p DIALOG -c > /dev/null
/usr/sbin/sfs_load: line 2502: 10050 Terminated $GTKDIALOG -p DIALOG -c > /dev/null
sfs_load: --stop
#
#
# make
make: *** No targets specified and no makefile found. Stop.
#
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9022

PostPosted: Sat 21 Dec 2013, 14:26    Post subject:  

Thanks for the input.

So that just demonstrates the extracting of an sfs to a full install...which is what i assume is normally offered. The idea is not to add to or interfere with the full install...plus its a bit on the clumsy side.

Wanted some confirmation that a sfs can be somehow unioned to / for a full install.

tested in lucid...union of sfs and / to /...well that failed

tried using a aufs mount in a similar fashion...ie union to 'union' including '/' as the bottom layer...failed also. So really in this case aufs refuses to include '/' as a layer ...question is can later kernels do this.?

By the way has no problem adding the sfs to /usr with
busybox mount -t aufs -o udba=reval,diropq=w,dirs=/root/app=ro:/usr=ro unionfs /usr
in other words you can union to one of the layers but obviously no use for the hundreds of sfs files built to run on '/'

mike
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9022

PostPosted: Sat 21 Dec 2013, 14:38    Post subject:  

busybox mount -t aufs -o udba=reval,diropq=w,dirs=/=rw:/root/app=ro unionfs /
where /root/app has had the sfs mounted already.

that sort of thing...but apart from the upsidedown layering would / be ok for the read/write layer (something would have to be ) ? Also if it did succeed its unlikely it could be unmounted...my /usr mount certainly would not.

mike
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5043
Location: Arizona, U.S.A.

PostPosted: Sat 21 Dec 2013, 15:01    Post subject:  

You can`t union on / of course, so it would have to be a boot process thing probably, not an add-on.

Like Puppy, make the initial union mount on /U with / as the top Save layer and chroot into /U.
Now you can add and subtract SFS files with sfs_load. ( I wish it had been made to do any image file )
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9022

PostPosted: Sat 21 Dec 2013, 18:48    Post subject:  

Quote:
You can`t union on / of course, so it would have to be a boot process thing probably, not an add-on.

your first post seemed to suggest otherwise...not to worry..I double checked anyway.

Quote:
Like Puppy, make the initial union mount on /U with / as the top Save layer and chroot into /U.
Now you can add and subtract SFS files with sfs_load. ( I wish it had been made to do any image file )

a layered variation...not the purpose here.

This is a way of using an sfs with a standard (not just puppy really) full install. It appears to work... seems simple enough ...in fact the only fiddly bit I can see is finding a way to create a menu entry that would be for a universal script rather than a one script per app situation.... Also note 'app' can be a normal file tree, or an image file or whatever...

here's the steps in bash... and unionfs-fuse attached for anyone interested in testing it out....used xchat as example sfs. and testing in /root . note how the unionfs-fuse command wants full paths. originally fakechroot was used though it seems to work with just chroot...I am sure there are reasons to use the latter but this seems ok for testing purposes...means the only binary to add is unionfs-fuse making it applicable to any puppy.

Code:
mkdir -p union
mkdir -p app
mkdir -p write

mount -o loop xchat.sfs app
unionfs-fuse -o nonempty -o allow_root -o cow /root/write=RW:/root/app=RO:/=RO /root/union

chroot union xchat


mike
unionfs-fuse.tar.gz
Description  place in /usr/bin
gz

 Download 
Filename  unionfs-fuse.tar.gz 
Filesize  16.5 KB 
Downloaded  235 Time(s) 
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5043
Location: Arizona, U.S.A.

PostPosted: Sat 21 Dec 2013, 20:51    Post subject:  

Yes, I`ve made union versions of my AppPkg.

My early post I ment unioning a SFS file or / ( as layers ), not unioning on / ( Sorry...).

If you mount on a dir that`s not empty it`s contents disappears.
unionfs-fuse will do this, your example uses the option but it`s not needed as the dir is empty.

I found I had to bind mount some of Linux`s mounted dirs to get it to work ( as amigo said ).
I also found that mhddfs worked very easily with no fussing or binding. It uses libfuse.
But as amigo pointed out, it`s not made for this purpose. But if it works... Go with it.!

You can make a chroot env. after booting and add/remove SFS files.
Each of the SFS apps running in the chroot, the rest of the Full install running outside.
I`ve never tried to add/remove SFS files using unionfs-fuse, but I`d think it`d do it.
.
Back to top
View user's profile Send private message 
jamesbond

Joined: 26 Feb 2007
Posts: 2232
Location: The Blue Marble

PostPosted: Sun 22 Dec 2013, 03:35    Post subject:  

mikeb wrote:
busybox mount -t aufs -o udba=reval,diropq=w,dirs=/=rw:/root/app=ro unionfs /
where /root/app has had the sfs mounted already.
Great idea but wrong implementation.
1. Aufs will fail because "/" is already aufs (aufs can't have another aufs as its branch, unless you force it to do so in kernel-config - but I reckon this isn't a good idea anyway).
2. But you can insert (or delete) another branch into existing aufs mounts (including /): mount -t aufs -o append:/root/app=ro unionfs /.
This line, in fact, is the magic how various implementations of "SFS-On-The-Fly" works.

Quote:
that sort of thing...but apart from the upsidedown layering would / be ok for the read/write layer (something would have to be ) ? Also if it did succeed its unlikely it could be unmounted...my /usr mount certainly would not.
Yes it can be unmounted if you can ensure that no applications are holding a file on the layer you want to unmount. Aufs also has the option to *remove* layers, and once the layer is removed you can unmount the sfs.
3. mount -t aufs -o del:/root/app unionfs / (and then umount /root/app)

Unionfs-fuse doesn't have the same limitation, and it can be used as non-root. But the "chroot" needs root privilege anyway (unless if you use fakechroot - but fakechroot isn't 100% foolproof). Whichever works, I suppose.

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2290

PostPosted: Sun 22 Dec 2013, 08:04    Post subject:  

I used fakechroot with unionfs-fuse with a mind to non-root users. unionfs-fuse has a 'chroot' option but I have not played with it all. Some of you youngsters might see how that goes.

A full implementation with mount-binded dirs would still need 'bindfs' which is another FUSE tool which implements 'mount --bind' capability.

Yes, aufs will restrict you -and rightfully so, I believe. The title says full install, though. Does that really mean there is no aufs being used?

If not, then you can do whatever you like with either aufs or uniofs-fuse. The idea with the chrome app using unionfs-fuse/fakechroot was to come up with a solution att eh app level, so that it could be used anywhere and would self-load. The effect of sandboxing by using chroot is just an extra possible benefit.

You could use unionfs-fuse to union '/' if you wanted to. It's just that that has to be done from within an initrd -before that 'real', but layered, root is mounted. No user would ever be aware of any I/O latency, since it is still faster than disk I/O.

The last thing is security. Yeah, there are ways to break out of a chroot, but this implementation minimizes the possibility of that since the whole chroot is done in one command. Compare with setting up a union manually, and then chrooting in there with:
'chroot union-dir /bin/bash'
Now, that one is 'dangerous', if you like. But the end result can be no worse than allowing a mortal to simply run as root...

You can easily (well, here at least) use a layer for a shared /usr or similar setup. Or you can create a completely different distro all under a subdir somewhere and just chroot in there for a completely different runtime. In such a case, you should probably unmount both /proc and /sys before entering the chroot and then re-mount them within the chroot.

Also, I want to stress again, mhddfs is fundamentally different from aufs or unionfs-fuse. With mhddfs, *all* layers are presumed to be writable. mhddfs works like a distributed volume. When new files are added you cannot control directly which part of the volume they will be written to. It will always write to the 'layer' which has the most free space. It does have a simpler syntax, but is not really the right tool for this kind of application.

In the script for the chrome app there are notes about the mount locations, which you said you were still confused about. Since I have a true multi-user system I pay attention to these things... I avoided using /tmp completely because anything there is accessible by anyone. Our mortal wants a bit of privacy. So, the only other place we can assume that a mortal has write privileges is under his/her $HOME directory(see POSIX specs). What I did for the chrome app was create a dir under their $HOME, but not directly under HOME. We create a dir like would be had for user configuration files for any other program, and then do our mounting under there -as well as putting the write layer there -that keeps everything under one dir. Sine the chrome thing was packaged as a Rox AppDir, I used $HOME/Choices as the top subdir. But one could use:
$HOME/.config/program-name
Avoid other locations for best compatibility and to keep non-angry
users.
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9022

PostPosted: Sun 22 Dec 2013, 08:26    Post subject:  

Quote:
Great idea but wrong implementation.
1. Aufs will fail because "/" is already aufs (aufs can't have another aufs as its branch, unless you force it to do so in kernel-config - but I reckon this isn't a good idea anyway).
2. But you can insert (or delete) another branch into existing aufs mounts (including /): mount -t aufs -o append:/root/app=ro unionfs /.
This line, in fact, is the magic how various implementations of "SFS-On-The-Fly" works.

That was a semi joke funny as it was suggested that the union could be made from / to / .... just to confirm that this need has not already been satisfied with some kernel update.

I have a sfs on the fly script that's a fraction of the size everyone else is using and layers in the correct order. It also unmounts on the fly. ..fast and reliable with no problems with files being hidden. Based on the slax way rather than puppy.
This is not really a post about aufs...that was a side track...and multiple sfs layering for frugal distros is something I have been using for years...so lets drop that one. Smile

nonempty I assume it for the possibly persistant write layer.

chroot on puppy since its running as root....fake if not...that clarifies that one.

My tests yesterday needed no bind mounting ...just ran as is.
....that was with a full install... perhaps doing it on an already unioned filesystem is another matter.

@amigo... thought you might drop in at some point...just as I was writing this in fact Very Happy
I started this as your union-fuse method seemed a neat way of using sfs when a full install is involved... and yes no aufs whatsoever. As mentioned no binding seems necessary ... just need a neat way of launching required.
I was not thinking particularly in terms of security or privacy
but living in home and fakechroot would indeed do the job. Otherwise for puppy and its open legs, /tmp could be used for a total clean up after use. My only thought was a persistant 'write' would be more convenient.

Apart from utilising sfs files it could be used for running software over a network.

Its really about protecting those vunerable full installs and speed. Something to chew on so a fresh post with your info seemed appropriate.... someone might take up the idea.

mike
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2290

PostPosted: Sun 22 Dec 2013, 11:52    Post subject:  

Well, if you are not using aufs otherwise, it would certainly do the job and fits well if you have/want a managed system. I like the idea of individual chroots and it's a good selling point.
Oh, about fakechroot, if your /bin/sh points to busybox you'll need to patch the getopt out of the wrapper script.
Back to top
View user's profile Send private message 
mikeb


Joined: 23 Nov 2006
Posts: 9022

PostPosted: Sun 22 Dec 2013, 11:58    Post subject:  

Quote:
Oh, about fakechroot, if your /bin/sh points to busybox you'll need to patch the getopt out of the wrapper script

yes discovered that one Very Happy

mike
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2290

PostPosted: Sun 22 Dec 2013, 12:47    Post subject:  

I thought I had patched it out, but I sure don't find that right now.... If you did, be a good fellow and post a patch.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 2 [22 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1075s ][ Queries: 13 (0.0072s) ][ GZIP on ]