Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 26 Nov 2014, 04:05
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Savefolder without 'mount -o bind' - works
Moderators: Flash, Ian, JohnMurga
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 8 Posts_count   Goto page: 1, 2, 3, ..., 6, 7, 8 Next
Author Message
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Fri 03 Oct 2014, 13:21    Post_subject:  Savefolder without 'mount -o bind' - works
Sub_title: Use symbolic links instead
 

The following are patches to "init" and "rc.shutdown" against TahrPup 5.8.3.rc1,
that implement 'savefolder' using symbolic links instead of 'mount -o bind'.

Patch 1) "init" script in "initrd.gz":
Code:
--- init.orig   2014-10-04 01:58:54.960910335 +1000
+++ init   2014-10-04 02:26:50.898497667 +1000
@@ -1078,7 +1078,10 @@
      ;;
    esac
   fi
-  if [ "$CRYPTO" != "" ] ; then
+  if [ -d /mnt/dev_save$PUPSAVEFILE ]; then
+   rm -r -f $CREATEPUPSAVE2FS
+   ln -s /mnt/dev_save${PUPSAVEFILE} $CREATEPUPSAVE2FS
+  elif [ "$CRYPTO" != "" ] ; then
    echo "" >/dev/console
    echo "Mounting encrypted $PUPSAVEFILE..." > /dev/console
    while true; do
@@ -1124,8 +1127,6 @@
      losetup -d /dev/loop1
     fi
    done
-  elif [ -d /mnt/dev_save$PUPSAVEFILE ]; then
-   mount -o bind /mnt/dev_save${PUPSAVEFILE} $CREATEPUPSAVE2FS
   else #pupsave not encrypted.
    # if pupsave increase requested
    if [ -f /mnt/dev_save/pupsaveresizenew.txt ];then #131225 changed to function
@@ -1967,6 +1968,12 @@
 do
  mount -o move $ONEMNT /pup_new/initrd${ONEMNT}
 done
+if [ ${PUPSAVEFILE} -a -d /pup_new/initrd/mnt/dev_save${PUPSAVEFILE} ]; then
+ if [ ${CREATEPUPSAVE2FS} -a -L ${CREATEPUPSAVE2FS} ]; then
+  rm -r -f /pup_new/initrd${CREATEPUPSAVE2FS}
+  ln -s ./mnt/dev_save${PUPSAVEFILE} /pup_new/initrd${CREATEPUPSAVE2FS}
+ fi
+fi
 
 #v4.02 bring back, but allocate more space (/4 instead of /8)....
 #v3.97 a problem can run out of /tmp space, remove...

To provide access to the save directory during "init", "/pup_rw" is changed to be a symbolic link to the save directory.
To provide direct access to the save directory after the switch root, "/pup_new/initrd/pup_rw" is changed to be a symbolic link to the save directory.
Fortunately 'aufs' cooperates nicely, when the specified branch is a symbolic link, it follows the link and uses the real path to the directory.
Code:
# cat /sys/fs/aufs/si_*/br0
/initrd/mnt/dev_save/tahr/tahrsave=rw


I've also changed the logic from:
Code:
if savefile is encrypted #assume is a file
else if savefile is a directory #assume no encryption
else #is unencrypted file
to
Code:
if savefile is a directory #could include encrypted option
else if savefile is encrypted #must be a file
else #is unencrypted file


Patch 2) "rc.shutdown":
Code:
--- rc.shutdown.orig   2014-08-28 06:07:44.000000000 +1000
+++ rc.shutdown   2014-10-04 02:04:09.922537392 +1000
@@ -440,12 +440,11 @@
   [ "`echo -n "$SAVEFILE" | grep "2fs"`" != "" ] && FILEFS="ext2"
   [ "`echo -n "$SAVEFILE" | grep "4fs"`" != "" ] && FILEFS="ext4" #120427 01micko
 
-  if [ "$CRYPTO" = "" ];then
-   if [ -d $SMNTPT$SAVEFILE ]; then
-    mount -o bind $SMNTPT$SAVEFILE /tmp/save1stpup
-   else
-    mount -t $FILEFS -o noatime,rw,loop $SMNTPT$SAVEFILE /tmp/save1stpup
-   fi
+  if [ -d $SMNTPT$SAVEFILE ]; then
+   rm -r -f  /tmp/save1stpup
+   ln -s $SMNTPT$SAVEFILE /tmp/save1stpup
+  elif [ "$CRYPTO" = "" ];then
+   mount -t $FILEFS -o noatime,rw,loop $SMNTPT$SAVEFILE /tmp/save1stpup
   else
    #note: loop2 is kept available for scripts to use. but, do this to find free one...
    DEVLOOP="`losetup-FULL -f`"
@@ -488,7 +487,7 @@
    [ "$BASENAME" = "root" ] && cp -a $ONEDIR/.[0-9a-zA-Z]* /tmp/save1stpup/${BASENAME}/ #v2.16exp4
   done
   sync
-  umount /tmp/save1stpup
+  [ -L /tmp/save1stpup ] || umount /tmp/save1stpup
 
   #w481 if low-ram, save-file is on a fast media, and no swap file/partition, then create one...
   swapfile_func $SMNTPT $SAVEPART
@@ -675,8 +674,10 @@
    sync
   fi
  fi
- busybox mount -t $SAVEFS -o remount,ro $SAVEDEV /initrd${SAVE_LAYER} 2>/dev/null
- umount-FULL -i -n -l /initrd${SAVE_LAYER} 2>/dev/null #-l is lazy unmount.
+ if [ $SAVEDEV ]; then
+  busybox mount -t $SAVEFS -o remount,ro $SAVEDEV /initrd${SAVE_LAYER} 2>/dev/null
+  umount-FULL -i -n -l /initrd${SAVE_LAYER} 2>/dev/null #-l is lazy unmount.
+ fi
 fi
 
 #v2.16 try one more thing for ntfs... lazy unmount, suggested by GuestToo...

At first shutdown a symbolic link is created to the actual save directory so the copying code will work.
After prepending the dummy rw layer to the aufs stack, only do the mount commands for a savefile, there is no mount corresponding to a savefolder.

Note: If only we had implemented it this way in the first place, we might have avoided the headaches caused by 'mount -o bind'.

gyro

Edit: The code above is a fix to the 'savefolder' facility that already exists in TahrPup. I intend to provide an example for a puppy with no existing 'savefolder' facility.

Edit2: An "initrd.gz" for TahrPup 5.8.3.rc1.sfs can be downloaded from here http://www.fishprogs.info/puppy/tahr/initrd.gz
A ydrv for TahrPup containing "rc.shutdown" and "sfs_load" can be downloaded from here http://www.fishprogs.info/puppy/tahr/ydrv_tahr_5.8.3.rc1.sfs

Edited_times_total
Back to top
View user's profile Send_private_message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11164
Location: Arizona USA

PostPosted: Fri 03 Oct 2014, 16:08    Post_subject:  

Will this work with a multisession Puppy DVD?
Back to top
View user's profile Send_private_message 
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Fri 03 Oct 2014, 16:28    Post_subject:  

Flash wrote:
Will this work with a multisession Puppy DVD?
No.
'savefolder' requires that saving is done on a Linux partition.

gyro
Back to top
View user's profile Send_private_message 
mavrothal


Joined: 24 Aug 2009
Posts: 1803

PostPosted: Fri 03 Oct 2014, 16:52    Post_subject: Re: Savefolder without 'mount -o bind' - works
Sub_title: Use symbolic links instead
 

gyro wrote:

The code above is a fix to the 'savefolder' facility that already exists in TahrPup. I intend to provide an example for a puppy with no existing 'savefolder' facility.


Have a test ISO?

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send_private_message 
starhawk

Joined: 22 Nov 2010
Posts: 3143
Location: Everybody knows this is nowhere...

PostPosted: Fri 03 Oct 2014, 19:28    Post_subject:  

I remember that there was an option to do a sort of hybrid install where the install itself was frugal but you could save to / and (theoretically) not have to bother with savefiles filling and all -- it would fill the remaining space in the root partition with savefile goodness automatically, or something like that.

Unfortunately it was abandoned before it became functional, and eventually it got removed. The few times I've tried it, as soon as I rebooted from the first boot, Puppy would make a mess on the carpet and generally not work. I don't recall the exact details but it was lots of red text having to do with one of the pup_ro bits (I want to say ro2?) followed by a kernel panic.

I would love to see that option actually made functional and brought back as such. Would make Puppy just that much more easy to use! (I wonder if it could be implemented simply as "put the savefile in / rather than a folder, and fill the entire partition with it"?)

_________________

Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4441
Location: Earth

PostPosted: Fri 03 Oct 2014, 19:35    Post_subject:  

Great. Seemingly plays into the hands of "remaster this very system".
_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
mikeb


Joined: 23 Nov 2006
Posts: 8688

PostPosted: Sat 04 Oct 2014, 06:46    Post_subject:  

bind mount was used for the save folder to avoid altering the pup_rw folder arrangement to keep compatability. Also its puppies ill thought through scripts unable to deal with it that was the problem rather than something wrong with the technique itself.

With any addition/modification to save techniques the designer has to try and keep everything as-is as much as possible otherwise you would need to change the name to Porteus or something Very Happy
I dynamically created additional folders for extra sfs but still keep pup_ro2 for exactly the same reasons. Certainly rearranging the initrd structure would make life a whole lot easier at times but compromise is the key word here.

mike
Back to top
View user's profile Send_private_message 
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Sat 04 Oct 2014, 11:02    Post_subject:  

mikeb wrote:
bind mount was used for the save folder to avoid altering the pup_rw folder arrangement to keep compatability. Also its puppies ill thought through scripts unable to deal with it that was the problem rather than something wrong with the technique itself.

Yes, at the time I couldn't work out how the single bind mount could be replaced with a single symbolic link.
I still can't, but in this solution, there are 2 symbolic links, one that works during 'init' and another one that works for the running system, after the switch root.

gyro
Back to top
View user's profile Send_private_message 
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Sat 04 Oct 2014, 11:05    Post_subject:  

starhawk wrote:
(I wonder if it could be implemented simply as "put the savefile in / rather than a folder, and fill the entire partition with it"?)
And why can't the contents of a directory "fill the entire partition"?

gyro
Back to top
View user's profile Send_private_message 
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Sat 04 Oct 2014, 11:10    Post_subject: Re: Savefolder without 'mount -o bind' - works
Sub_title: Use symbolic links instead
 

mavrothal wrote:
Have a test ISO?
Hmm, I seem to remember you asking me this question in a different place.
The answer is the same, No.
However I plan to make an initrd.gz and a ydrv available for download.

gyro
Back to top
View user's profile Send_private_message 
starhawk

Joined: 22 Nov 2010
Posts: 3143
Location: Everybody knows this is nowhere...

PostPosted: Sat 04 Oct 2014, 11:20    Post_subject:  

gyro wrote:
starhawk wrote:
(I wonder if it could be implemented simply as "put the savefile in / rather than a folder, and fill the entire partition with it"?)
And why can't the contents of a directory "fill the entire partition"?

gyro


They can. I'm just working with what I'm used to Wink I rather liked the idea of that feature (frugal with save to root)... like I said, would make life with Puppy that much easier... Wink

_________________

Back to top
View user's profile Send_private_message 
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Sat 04 Oct 2014, 12:33    Post_subject: initrd.gz and ydrv for TahrPup 5.8.3.rc1  

Url's for downloading an initrd.gz and ydrv for TahrPup 5.8.3.rc1 are now included at the end of the first post.
The ydrv contains sfs_load, to avoid partitions remaining mounted on first boot. (It's fixed in QuickPet, but that's too late for first boot.)

gyro
Back to top
View user's profile Send_private_message 
mavrothal


Joined: 24 Aug 2009
Posts: 1803

PostPosted: Sat 04 Oct 2014, 16:25    Post_subject: Re: Savefolder without 'mount -o bind' - works
Sub_title: Use symbolic links instead
 

gyro wrote:
Note: If only we had implemented it this way in the first place, we might have avoided the headaches caused by 'mount -o bind'.


So is this working without all the other changes implemented so the current "bind" method would work properly? Would it work say, on Slacko 5.7 or PrecisePup properly without additional changes (other than the shutdown config)?

BTW, which would you say is the advantage of the symlink method over the bind method. Which current or future problem is it addressing better? (I would think device management by different file managers maybe. Does it?)

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send_private_message 
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Sun 05 Oct 2014, 03:47    Post_subject: Re: Savefolder without 'mount -o bind' - works
Sub_title: Use symbolic links instead
 

mavrothal wrote:
So is this working without all the other changes implemented so the current "bind" method would work properly? Would it work say, on Slacko 5.7 or PrecisePup properly without additional changes (other than the shutdown config)?
Yes, that is my contention.
I'm currently working on porting it to Dpup Wheezy 3.5.2.11, without any of the additional changes. I've got a patched 'init' and 'rc.shutdown' working, I just need to work out the best way to incorporate, an updated 'shutdownconfig' so a savefolder can be created at first shutdown. (I currently create a small savefile and then use my 'savefile2dir' utility to convert it to a savefolder.)
I will be posting the files for this port as an example, in this topic.

mavrothal wrote:
BTW, which would you say is the advantage of the symlink method over the bind method. Which current or future problem is it addressing better? (I would think device management by different file managers maybe. Does it?)
Any code that does not appropriately handle the existence of more that 1 mount entry for a single device, will not get broken by the symlink method.
A simple example is "df", using the symlink method, it works as expected in TahrPup.
Things like 'PcmanFM' should work without any modification.
The long term advantage is that we don't know if there is more software out there that would fail if it was run in an environment with more than 1 mount entry per device.

gyro
Back to top
View user's profile Send_private_message 
gyro

Joined: 28 Oct 2008
Posts: 522
Location: Brisbane, Australia

PostPosted: Sun 05 Oct 2014, 04:59    Post_subject:  

I have attached the diff files I used for Dpup Wheezy 3.5.2.11.
I have also attached my utility for converting an existing savefile to a savefolder.
The files are really gzipped, so download and move them to where you want them and then gunzip them.

I post these as an example of porting this savefolder facility to puppies that are not based on a current version of woof-ce, as TahrPup is, and hence do not contain any savefolder facility.

Warning, while this version of savefolder avoids the "more than 1 mount entry" problems that plagued the development of the bind mount version, it does not make any difference to other problems.
Example: when I previously ported savefolder to Dpup Squeeze Exprimo, the "rc.shutdown" would fail. I expect that this version could have the same problem.

Note: Once I have sorted out the easiest way to include an updated 'shutdownconfig' I intend to upload a patched 'initrd.gz' and a 'ydrv_wheezy_3.5.2.11.sfs' containing 'rc.shutdown' and 'shutdownconfig'.

Note2: I have included 'savefile2dir' because without an updated 'shutdownconfig' you can't create a savefolder. But this utility can convert an existing savefile to a savefolder. So testing of the savefolder facility can begin once you have a successfully patched 'initrd.gz' and 'rc.shutdown'.

gyro
savefile2dir.gz
Description  utility, executable bash script.
gz

 Download 
Filename  savefile2dir.gz 
Filesize  1.61 KB 
Downloaded  41 Time(s) 
rc.shutdown.diff.gz
Description  diff file for 'rc.shutdown'
gz

 Download 
Filename  rc.shutdown.diff.gz 
Filesize  720 Bytes 
Downloaded  28 Time(s) 
init.diff.gz
Description  diff file for 'init' script
gz

 Download 
Filename  init.diff.gz 
Filesize  1.15 KB 
Downloaded  30 Time(s) 
Back to top
View user's profile Send_private_message 
Display_posts:   Sort by:   
Page 1 of 8 Posts_count   Goto page: 1, 2, 3, ..., 6, 7, 8 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1087s ][ Queries: 12 (0.0043s) ][ GZIP on ]