Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 25 Oct 2014, 10:38
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Mounting and Linking SFS files for use
Moderators: Flash, Ian, JohnMurga
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 3 Posts_count   Goto page: 1, 2, 3 Next
Author Message
jrb


Joined: 11 Dec 2007
Posts: 1040
Location: Smithers, BC, Canada

PostPosted: Tue 15 Sep 2009, 13:57    Post_subject:  Mounting and Linking SFS files for use
Sub_title: Alternate to loading sfs with aufs
 

12Nov09,
(I have uploaded SFS_Linker-431bk-1.1.pet for use in Puppy431 andSFS_Linker-no_def-1.1.pet which does not have modified /usr/local/defaultprograms for people who wish to have SFS_Linking without changing the desktop icon or MIME-type behavior.

These are the latest versions of SFS_Linker and seem to be preforming well.)


I have been looking for alternatives to load_sfs as used in ChoicePup412. Although it works well, it only works with the aufs1 file layering system.

The new kernels that Barry is introducing into Puppy4.3 use aufs2 which fails to mount the sfs's even if they're sfs4.

Looking at an idea put forward by sunburnt and reintroduced by trio for copying sfs into full installs I thought it might be possible to link the files in the mounted sfs to / rather than copying them. This would take no space in a frugal installs pup_save file.

I copied some of trio's code and put together this script:
Code:
#!/bin/sh
#SFS linker

SFSNAME=`basename $1`

   mkdir /mnt/"$SFSNAME"
   mount -t squashfs "$1" /mnt/"$SFSNAME" -o loop
   cp -rs /mnt/"$SFSNAME"/* / #jrb-add -rs, remove -a symlink option 15sep09
   
   cd /mnt
   APATTERN="s/.\/$SFSNAME//g"
     find ./$SFSNAME -type f -mount -mindepth 2 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' > /root/.packages/$SFSNAME.files
     sync
     find ./$SFSNAME -type l -mount -mindepth 2 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' >> /root/.packages/$SFSNAME.files
     sync
   echo -e "\"${SFSNAME}\" \"${SFSNAME} : \" on \" \" \\"   >> /root/.packages/alienpackages.txt
   
   mkdir -p /root/my_links/linked_sfs
   ln -s $1 /root/my_links/linked_sfs/

   ldconfig
   fixmenus
   Xdialog --title "sfs_linker" --msgbox "Success! `basename $1` loaded.\n"  6 0
   jwm -restart


I have not built an uninstall script yet. You can remove the symlinks in petget manager although it leaves behind empty folders. You have to go to /mnt and unmount the sfs files and then delete their folders.

It works with most of the ch-sfs packages. Noticeable exceptions are the mozilla browsers and openoffice. I have a feeling that the $HOME variable my need to be set. ch-GoogleEarth-4.3.sfs opens but doesn't work.

Any advice or suggestions will be greatly appreciated (I still consider myself a linux noob so keep it simple).

This is for use in Puppy412 or 421. The petget manager is different in Puppy4.3, will have to rewrite for that.

If you feel brave start a new pup_save and try it out. Right click on an SFS, choose OpenWith-sfs_linker.

Thanks, J

Edited_times_total
Back to top
View user's profile Send_private_message 
seaside

Joined: 11 Apr 2007
Posts: 887

PostPosted: Tue 15 Sep 2009, 18:34    Post_subject:  

Jrb,

I tried it with ch-mtpaint and got these errors ..

Code:
# sfslinker ch-mtpaint-3.30.sfs
cp: cannot create symbolic link `/root/Choices/MIME-types/image_bmp' to `/mnt/ch                                                              -mtpaint-3.30.sfs/root/Choices/MIME-types/image_bmp': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_gif' to `/mnt/ch                                                              -mtpaint-3.30.sfs/root/Choices/MIME-types/image_gif': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_jpeg' to `/mnt/c                                                              h-mtpaint-3.30.sfs/root/Choices/MIME-types/image_jpeg': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_png' to `/mnt/ch                                                              -mtpaint-3.30.sfs/root/Choices/MIME-types/image_png': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_tiff' to `/mnt/c                                                              h-mtpaint-3.30.sfs/root/Choices/MIME-types/image_tiff': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_x-pixmap' to `/m                                                              nt/ch-mtpaint-3.30.sfs/root/Choices/MIME-types/image_x-pixmap': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_x-png' to `/mnt/                                                              ch-mtpaint-3.30.sfs/root/Choices/MIME-types/image_x-png': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_x-xpixmap' to `/                                                              mnt/ch-mtpaint-3.30.sfs/root/Choices/MIME-types/image_x-xpixmap': File exists
cp: cannot create symbolic link `/usr/local/lib/X11/mini-icons/mini-camera.xpm'                                                               to `/mnt/ch-mtpaint-3.30.sfs/usr/local/lib/X11/mini-icons/mini-camera.xpm': File                                                               exists
cp: cannot create symbolic link `/usr/local/lib/X11/mini-icons/mini-palette.xpm'                                                               to `/mnt/ch-mtpaint-3.30.sfs/usr/local/lib/X11/mini-icons/mini-palette.xpm': Fi                                                              le exists
cp: cannot create symbolic link `/usr/local/lib/X11/pixmaps/paint48.png' to `/mn                                                              t/ch-mtpaint-3.30.sfs/usr/local/lib/X11/pixmaps/paint48.png': File exists
find: warning: you have specified the -mount option after a non-option argument                                                               -type, but options are not positional (-mount affects tests specified before it                                                               as well as those specified after it).  Please specify options before other argum                                                              ents.

find: warning: you have specified the -mindepth option after a non-option argume                                                              nt -type, but options are not positional (-mindepth affects tests specified befo                                                              re it as well as those specified after it).  Please specify options before other                                                               arguments.

find: warning: you have specified the -mount option after a non-option argument                                                               -type, but options are not positional (-mount affects tests specified before it                                                               as well as those specified after it).  Please specify options before other argum                                                              ents.

find: warning: you have specified the -mindepth option after a non-option argume                                                              nt -type, but options are not positional (-mindepth affects tests specified befo                                                              re it as well as those specified after it).  Please specify options before other                                                               arguments.



Despite the errors, everything seems to be working as usual. Very Happy
Back to top
View user's profile Send_private_message 
jrb


Joined: 11 Dec 2007
Posts: 1040
Location: Smithers, BC, Canada

PostPosted: Tue 15 Sep 2009, 20:11    Post_subject:  

Thanks seaside,
It's trying to create symlinks in folders where a file with that name already exists. I hadn't realized all those mimetypes were in the mtpaint package.

Just glad to see that the originals weren't removed when I uninstalled with petget manager. They may well have been if they had been placed there with another .pet. I think this method may require a bit more care and testing when putting together the sfs's.

I haven't tried all the sfs's but so far opera, flash, geany, dillo, pnethood, cups, wine, boost, pwidgets, mplayer, gxine, and mtpaint are all working well, very promising. Very Happy
Back to top
View user's profile Send_private_message 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Tue 15 Sep 2009, 23:05    Post_subject:  

jrb; What I have concluded is that temp. files and links should go in the /tmp dir.
As in usb-auto, I make the dir.: /tmp/usb-auto to hold the links.
The next time Puppy boots, the links and dir. should all be gone.
Self cleaning... At least Puppy use to work this way, things keep changing.

Also rather than relying entirely on links, use PATH & LD_LIBRARY_PATH.
I made a GUI to manage the paths into the Squash file`s mount points.

I suggested apps. that will run mounted but not unioned be used that way.
Gimp only needed it`s config. file copied to /etc and it ran just fine!
With help from some Gurus like MU etc. more could be made to work.
Best to do the really important apps. first; Web, Media, Games, etc.

SFS files are nice as they don`t corrupt or take up space in the Save file.
A new extension is needed for them, like: ".sfa" (Squash File Attached).
With a mime file type, a simple click could attach & unattach the SFA file.
A new SFA file manager would be needed once there were lots of SFA files.

The SFA files don`t have to be made like SFS files are, they could be simpler.
A script inside the SFA file could setup the SFA file to work properly.
The script would also clean up when the SFA file was unmounted.
The script allows for handling the special setup needed by some apps.

SFA files are the first main step needed to get rid of the union file sys.
No union makes for a much simpler and more reliable OS.
The core OS can run in ram and the SFA files can be anywhere!
On a DVD, a HD, a USB, on the LAN, or on the Web! Web connect apps.!
Also the core OS can boot from anywhere and be distributed as well.
Back to top
View user's profile Send_private_message 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Fri 18 Sep 2009, 00:21    Post_subject:  

Hey jrb; Are you still working on this?
I have been thinking about it, and I have an idea for a SFA builder script.
It would install an app. package to a dir., then you can test it from there.
The apps. install dir. would act just like the SFA file`s mount point.
As links are made for the app., a file listing them is made in the SFA file.
Load and Unload the SFA file, the list is read and links made and deleted.
Once the app. is working from the dir., another script makes an SFA file.

Need shorter lengths and fewer additions to PATH and LD_LIBRARY_PATH.
So the SFA file would have just 1 dir. each: /bin, /sbin, /etc, /lib, and more.
Example: Only a /bin, not: /bin, /usr/bin, /usr/local/bin, so fewer paths.
And the files being only 1 dir. deep keeps the paths as short as possible.
With the paths set leading into the SFA file, only files that are referenced
with a full path need links, so fewer links need to be made and managed.

There`s other tricks that can be done also, but this adds good capability.
Back to top
View user's profile Send_private_message 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Fri 18 Sep 2009, 04:10    Post_subject:  

jrb; I was grabbing some more apps. for ChoicePup and thinking about this somemore...

How about making a small ram disk to hold all the links and use it as a union layer.
Then all the links are not cluttering up the Save file and don`t need to be cleaned up.
The links don`t take up any space, so a very small ram disk should do, ~ 16-32 MB ?
This still makes use of the FUSE union FS, but this is how ChoicePup still works.

I hope the use of links can be greatly reduced by the use of the 2 paths.
Imagine only 3 union layers! The Save file, the main SFS, and the links ram disk.
Less memory needed, and less cpu overhead to handle all those layers!
Back to top
View user's profile Send_private_message 
disciple

Joined: 20 May 2006
Posts: 6449
Location: Auckland, New Zealand

PostPosted: Fri 18 Sep 2009, 05:45    Post_subject:  

Quote:
Imagine only 3 union layers! The Save file, the main SFS, and the links ram disk.
Less memory needed, and less cpu overhead to handle all those layers!

That sounds great, but then I struggle to imagine anything Smile
Has anybody quantified the effect of using lots of union layers? Even a couple of dozen?
Have people even reported Puppy feeling slower if they add a couple of dozen .sfs addons?

_________________
DEATH TO SPREADSHEETS
- - -
Classic Puppy quotes
- - -
Beware the demented serfers!
Back to top
View user's profile Send_private_message 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Fri 18 Sep 2009, 12:30    Post_subject:  

disciple; Surely you don`t deny a union`s complexity and resources required?
At first it seems a great idea, but experience and understanding changes that.
Realize it`s unnecessary and cumbersome, there`s better methods to use.
### A better method:
App. Squash files are only mounted to the core OS running in memory.
OS dirs. that are written to often (/etc) are links pointing to dirs. in a Save file.
To change the core OS, make a new core SFS file, it`s small and quickly done.
Apps. in Squash files are the new package format, no installing, just use them.

Compared to Puppy or a Full Linux install... What a really great improvement!
Speed, simplicity, low cpu use, low memory use, modular packages and OS.
No installing-uninstalling apps. scattering files all through the OS, so messy!
Unix was great for a main frame admin., but Linux users change apps. about
as often as a 16 year old girl changes her clothes, managing the files is a pain.
That`s why ChoicePup is such a great Puppy build, it uses SFS files fully.
Back to top
View user's profile Send_private_message 
disciple

Joined: 20 May 2006
Posts: 6449
Location: Auckland, New Zealand

PostPosted: Fri 18 Sep 2009, 18:53    Post_subject:  

I'm not denying anything, I'm just wondering if anyone knows how much benefit this approach would have...
_________________
DEATH TO SPREADSHEETS
- - -
Classic Puppy quotes
- - -
Beware the demented serfers!
Back to top
View user's profile Send_private_message 
technosaurus


Joined: 18 May 2008
Posts: 4353

PostPosted: Fri 18 Sep 2009, 19:30    Post_subject:  

Gobolinux (and I think tiny core) use a similar method. I am testing a completely different method using ermine pro. It can pack an entire (uncompressed) filesystem into a single executable. Put the one executable in a sfs of the same name and modify the sfs mounter to try to execute a program of the same name as the sfs. Two clicks and you are there. The biggest downfall which is the same as sfs is dependencies. The great part about ermine pro is that it will automatically include all dependencies in the package unless you specify otherwise. Not too great for size if someone doesn't know how to package with it,but better for the user to wait an extra couple seconds on a download than have to track down dependencies.
_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send_private_message 
sunburnt


Joined: 08 Jun 2005
Posts: 5037
Location: Arizona, U.S.A.

PostPosted: Sat 19 Sep 2009, 00:20    Post_subject:  

technosaurus; I looked at GoboLinux years ago, and it`s rather complicated.
Rearranging the Linux dir. structure won`t make it better, it`s more complex.
The core OS here is Puppy`s 4 files, but no union FS complications.
The real change is to how the SFS files are used, and their structure.
Dependencies can always be in the SFS files of course, at any level.

disciple; The benefits are obvious, it`s simplicity is reason enough...
And the other obvious benefits are all the more reason to implement it.
We`ve had cars for 100 years now, but most see it as a bad thing...
To wonder if we should change to a better transit system is short sighted.
Back to top
View user's profile Send_private_message 
dejan555


Joined: 30 Nov 2008
Posts: 2689
Location: Montenegro

PostPosted: Sat 19 Sep 2009, 04:59    Post_subject:  

Haven't tried this on 4.3 but this is how rexterd does it on slitaz and he says it works on fatdog too:
http://murga-linux.com/puppy/viewtopic.php?p=330836#330836

_________________


Back to top
View user's profile Send_private_message Visit_website MSNM 
dejan555


Joined: 30 Nov 2008
Posts: 2689
Location: Montenegro

PostPosted: Sat 19 Sep 2009, 05:12    Post_subject:  

There;s similar command for merging branches in unionfs too, but for some reason doesn't apear to work, maybe needs some modification of init script or something else compiled in kernel:
http://www.linuxjournal.com/article/7714

So mounting on the fly is possible in both aufs and unionfs but for some reason don't work on puppy

_________________


Back to top
View user's profile Send_private_message Visit_website MSNM 
disciple

Joined: 20 May 2006
Posts: 6449
Location: Auckland, New Zealand

PostPosted: Sat 19 Sep 2009, 05:31    Post_subject:  

We're not talking about unioning on the fly, but about not unioning (well, reducing the number of unions, anyway).
_________________
DEATH TO SPREADSHEETS
- - -
Classic Puppy quotes
- - -
Beware the demented serfers!
Back to top
View user's profile Send_private_message 
dejan555


Joined: 30 Nov 2008
Posts: 2689
Location: Montenegro

PostPosted: Sat 19 Sep 2009, 05:38    Post_subject:  

I don't think more layers slows down anything but boot time
And I'm mentioning on the fly mounting cause it's related to topic

_________________


Back to top
View user's profile Send_private_message Visit_website MSNM 
Display_posts:   Sort by:   
Page 1 of 3 Posts_count   Goto page: 1, 2, 3 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1133s ][ Queries: 12 (0.0047s) ][ GZIP on ]