Enhance boot params, pupsfs, zdrv, fdrv, adrv, ydrv, psave

[gyro]

A project to enhance the facilites provided by the sfs boot parameters, pupsfs, zdrv, adrv, and ydrv. And to add a savefile/savefolder parameter psave.
It provides the facilities that might be expected from them, i.e. any of these sfs's can be specified to reside anywhere with any name.

Edit 9 Jun 2016:

Each parameter can now be specified as <partition>:<filename>.
Where <partition> can be specified as either a name (e.g. sdb4) or a Label (e.g. Work) or a UUID (e.g. 49baa82d-8c69).
If a Label or UUID is used only the beginning is needed, enough to make it unique.
If a partition is defined with "pupsys", this can be inherited by the other parameters. e.g if "pupsfs=Linux" is specified, "adrv=:pcmanfm.sfs" is equivalent to "adrv=Linux:pcmanfm.sfs".
If no <filename> is specified (e.g. sdb4 or Work:), the standard filename is used. (Trailing ':' is optional.)
If the <filename> does not exist, puppy will look for the standard filename.
If the <filename> begins with a "/", it is assumed to contain a full path, otherwise PSUBDIR is prepended to <filename>.
Puppy looks for the files on the specified partitions and only the specified partitions. So ensure any required partition is specified at least once.
If a specified file does not exist, the specification will be ignored.
If a specified file does exist, it takes precedence over the standard one.

Some things you can now do:
1) Specify the partition containing your frugal install with a Label or UUID, using "pupsfs". (Recommended).
2) Test a new adrv (etc..) that resides somewhere on a connected device, by specifying it as a boot parameter, knowing that if you get the specification wrong, puppy will use the standard one.
3) Have your install on an SSD but specify your savefolder on a different device, e.g. the HD.
4) Choose a particular savefolder amongst many by specifying it's filename using "psave".(Could be useful if your keyboard doesn't work during "init".)

Notes:
1) If "pmedia=usbhd" or "pmedia=usbflash" is specified, all ata partitions will be ignored.
So, if you want to specify both ata and usb partitions, ensure that "pmedia=atahd" or "pmedia=ataflash".
2) "shutdownconfig" requires patching to honour a "psave" specification during savefolder creation.

Using:
Download either the attached "init.woof.gz" or "init.gyro.gz",
gunzip the downloaded file and then rename it to "init",
then merge this "init" script into the "initrd.gz" of your puppy.
The "init.woof" file is the same as that uploaded into woof-ce.
The "init.gyro" is the version that I use, it has the following patches added:
"fsck4" - always fsck every ext4 file system before it is first mounted (pfix=fsck4),
"discard" - add the "discard" option when mounting an ext4 or f2fs file system on an SSD (pfix=trim),
"directory" - any of the "sfs's" can be a directory instead of an sfs (".sfs" is dropped from the standard names).
Such directories must reside in a Linux file system, and are never copied into ram.

Or download one of the following "initrd.gz" files, patched with "init.gyro" and use it in the appropriate puppy:
Tahrpup 6.0.5 "initrd.gz" from http://www.fishprogs.software/puppy/bpa ... /initrd.gz
Tahrpup64 6.0.5 "initrd.gz" from http://www.fishprogs.software/puppy/bpa ... /initrd.gz
Xenialpup 7.0.1 "initrd.gz" from http://www.fishprogs.software/puppy/bpa ... /initrd.gz

Also attached are patches for "shutdownconfig" and "rc.shutdown" to support the "psave" parameter.

The old patch files have been deleted, since they would no longer be complete.

Warning:
I have had some difficulties with "sfs_load" when specifying fdrv sfs's with arbitary names. "sfs_load" interpereted them as "extra sfs's".
This was improved by moving them to directories where "sfs_load" doesn't find them.

Old stuff follows:
----------------------------------------------------

Current limitations that get removed:
1) Only the pup..sfs and pupsave code is capbable of mounting a partition. All other sfs's must reside on one of these 2 partitions.
2) The current sfs parmaeters cannot work on usb drives.
3) Searching is either all partitions, or pdev1 and psavemark, or only all usb.
4) Sfs locations cannot be specified using partition Label's or UUID's.
5) The partition for a savefile/savefoler must be on the same device as the pup..sfs.

This project is implemented as 5 patches to the "init" script in "initrd.gz", (attached). That idea is that each patch could become a separate commit.
The patches are meant to be applied in sequence and sometimes depend on a previous patch, hence beginning with "1" "2"...
While each patch produces a working "init", implying that the sequence can be stopped at any point, the first 3 are a logical minimal set.
These patches are against "/woof-CE/woof-code/huge_extras/init", hence the "-woof" part of the patch filenames. Hopefully they will apply to recent "init" scripts.

Edit 31 May 2016:
The "1func-woof2.diff" patch remains unchanged.
Significant change is "2param-woof3.diff" to improve reliability.
Rest of patches are updated because line numbers are different.
Replaced "init" "init.woof3". This is the resultant file after applying all patches, and should be able to replace the "init" of any woof-ce puppy.
Added "init" "init.initrd_progs2". A patched "init" based on the "initrd_progs" project. This has been added because "1func-woof2.diff" fails to apply to the "initrd_progs" "init" script.
The tahr, tahr64, and xenial "initrd.gz" files have been updated.

Edit 26 May 2016:
The patches have all been replaced with "-woof2" pathces against "woof-CE/woof-code/huge_extras/init" testing branch as of 25/5/16.
This includes fdrv support.
I have also attached a patched version of this "init" script as "init.woof2". You should be able to use this as a replacement for the "init" script in the "initrd.gz" of any woof-ce based puppy.
The Thar and Tahr64 initrd.gz files are still notionally correct, since their original "init" scripts did not include fdrv support.

"1func-woof2.diff":
Creates new "find_onepupdrv" and "load_onepupdrv" functions that are used for all 4 sfs files.
Since the exact filename is always known, the actual "finding" is implemented as a simple file test.
The "load_onepupdrv" function is capable of mounting a partition, so any of the 4 sfs's can be on any partition.
All sfs files copied to ram now use a single tmpfs partition.
The size of 'init' is significantly reduced.

"2param-woof3.diff":
Changes the code supporting the sfs boot parameters to use the new "find" function.
This means that they can work for USB drives as well as ATA drives. And they will failover to the default files if the specified file does not exist.

"3search-woof3.diff"
Changes the searching method so that only partitions that are part of a boot param are searched.

"4id-woof3.diff"
Enables partitions to be specified by a partition Label or UUID.

"5psave-woof3.diff"
Adds a "psave" boot parmameter that can specify a partition and filename.
With patches 1,2,3 and 4 in place this is a small extension.

NOTE on the algorithm in "init":
While usb storage is settling, the internal (ata) partitions are searched.
When usb storage is ready, the usb partitions are searched.
The important complication is that up until usb storage is ready, the "init" code has no information about usb drvies and partitions. Hence the current sfs specifying code will fail to determine the fstype for usb partitions, and so fail.
In the new code running "blkid" before usb storage is ready, produces no information about usb drives, so it has to be run again after usb storage is ready.

gyro

[gyro]

The following boot paramaters:

Code: Select all

pmedia=atahd psubdir=/puppy/tahr ydrv=Work:pcmanfm32.sfs zdrv=sdb4:/zzdrv_tahr_6.0.5.sfs psave=smark

leads to the following PUPSTATE:

Code: Select all

PUPMODE=12
PDEV1='sdb2'
DEV1FS='ext4'
PUPSFS='sdb2,ext4,/puppy/tahr/puppy_tahr_6.0.5.sfs'
PUPSAVE='sdb4,ext4,/puppy/tahr/tahrsave'
PMEDIA='atahd'
#ATADRIVES is all internal ide/pata/sata drives, excluding optical, excluding usb...
ATADRIVES='sda sdb '
#ATAOPTICALDRIVES is list of non-usb optical drives...
ATAOPTICALDRIVES='sr0 '
#these directories are unionfs/aufs layers in /initrd...
SAVE_LAYER='/pup_rw'
PUP_LAYER='/pup_ro2'
#The partition that has the tahrsave file is mounted here...
PUP_HOME='/mnt/dev_save'
#(in /initrd) ...note, /mnt/home is a link to it.
#this file has extra kernel drivers and firmware...
ZDRV='sdb4,ext4,/zzdrv_tahr_6.0.5.sfs'
ADRV=''
YDRV='sdb2,ext4,/puppy/tahr/pcmanfm32.sfs'
#complete set of modules in the initrd (moved to main f.s.)...
ZDRVINIT='no'
#Partition no. override on boot drive to which session is (or will be) saved...
PSAVEMARK=''
PSUBDIR='/puppy/tahr'

Notes:
The ydrv is specified by a partition label, inherits psubdir.
The zdrv is specified by a partition name, does not inherit psubdir.
The save layer is specified by a partition label only, inherits psubdir and filename.

File "/initrd/tmp/puppy-file-search.log" shows that only sdb2 and sdb4 were serached:

Code: Select all

ALLDRVS=sda sdb sr0  
ONEDEV=sdb2 ONEFS=ext4 ONEDRV=sdb PSUBDIR=/puppy/tahr
  IGNORE=optical PSUBDIR=/puppy/tahr SAVEPART=sdb4 VMLINUZ=vmlinuz PDEV1=sdb2 PUPSFS=sdb2,ext4,/puppy/tahr/puppy_tahr_6.0.5.sfs
ONEDEV=sdb4 ONEFS=ext4 ONEDRV=sdb PSUBDIR=/puppy/tahr
  IGNORE=optical PSUBDIR=/puppy/tahr SAVEPART=sdb4 VMLINUZ=vmlinuz PDEV1=sdb2 PUPSFS=sdb2,ext4,/puppy/tahr/puppy_tahr_6.0.5.sfs

gyro

[gyro]

Some of these patches can be a bit of a pain to apply, depending upon the presence or absence of some recent changes to "init" in woof-ce.
So if you would like to try it on a puppy other than Tahr 6.0.5 or Tahr64 6.0.5, please ask here and I will try to produce an appropriate patched "initrd.gz" to add to those available for download via the first post.

gyro

[Robin2]

Does the download file "The Tahr64 6.0.5 "initrd.gz" I used for testing can be downloaded from" include the patches?

I have just tried it so I can answer my question - yes it does.

Even better it seems to have solved the problem I was looking for help for in this Thread http://murga-linux.com/puppy/viewtopic.php?t=106953

Thank you very much - really appreciated

...R

[Robin2]

While I can now get my ACER Cloudbook to boot TahrPup from a USB stick using a saveFile on the Solid State Drive I am still unable to make it boot TahrPup directly from the SSD. It gives the message "puppy_tahr64_6.0.5.sfs not found" even though that same file IS found (without needing any special parameters) when booting from the USB stick.

I can't figure which of the parameters I should use to tell it where to find the principal file. An example would be appreciated so that I can see the proper syntax.

Many thanks

...R

[ASD]

I'm pleased to read your good news. Again, try the SAVEMARK file I posted; it worked for me when I had a similar, though not identical, problem.

[gyro]

@Robin2,

Please post the boot parameters you used to get it to boot from the USB stick.
And post the boot parameters you are using to try to boot directly from the SSD.

You could try adding a "pupsfs=mmcblk0p2" boot parameter.

gyro

[Robin2]

The boot from the USB uses

Code: Select all

kernel vmlinuz
initrd initrd.gz 
append psave=mmcblk0p2

The unsuccessful boot from the SSD uses

Code: Select all

kernel vmlinuzTAHR64
initrd initrdTAHR64.gz 
append pupsfs=mmcblk0p2:/puppy_tahr64_6.0.5.sfs psave=mmcblk0p2

These are the same vmlinuz and initrd.gz with the names changed so they don't conflict with vmliuz for Quirky.

It would be helpful to have a brief description of what each of the parameters is intended to do.

For example does pmedia make it look at something or does it just prevent it from looking at other things?

What are adrv, ydrv and zdrv for?

What is psubdir for? How does it relate to pupsfs?

Again, many thanks.

...R

[gyro]

"psubdir" is use to specify the sub-directory that contains the puppy files.
So to avoid conflicts with Quirky, you could move all your tahr64 files into a sub-directory called something like "tahr64" perhaps, then rename "vmlinuz" and "initrd.gz" back to their default names.
Then you might boot with

Code: Select all

kernel /tahr64/vmlinuz
initrd /tahr64/initrd.gz
append pmedia=atahd psubdir=/tahr64 pupsfs=mmcblk0p2 psave=mmcblk0p2

The "init" script searches for "vmlinuz" by name. It also uses the "pmedia" to do different things depending on it containing "ata" or "usb" and also depending on it containing "hd" or "flash".
You should not need to specify the filename for "pupsfs" as that's the standard name any way.

"adrv", "ydrv" and "zdrv" are there to overide the standard filenames for those special ".sfs" files.
The "zdrv" file contains all the drivers, and so is normally present.
The "adrv" and "ydrv" are optional, and can be supplied to add extra files to puppy.

Note: With all your tahr64 puppy files in a single sub-directory you should not need any of these enhanced overide boot parameters. You shoud be able to boot with

Code: Select all

kernel /tahr64/vmlinuz
initrd /tahr64/initrd.gz
append pmedia=atahd psubdir=/tahr64 pdev1=mmcblk0p2

gyro

[jlst]

Very interesting. Some activity is happening at

https://github.com/puppylinux-woof-CE/initrd_progs

I'll add your patches in that repo and test

[Robin2]

Thanks @gyro for all your help.

I have the impression that your patches mean that the advice in this Thread http://www.murga-linux.com/puppy/viewtopic.php?t=35003 is not relevant.

What does the pdev1 parameter do?

I will do some more exploring about booting directly from the SSD but using the USB stick is a workable solution.

To add to the good news I have now been able to get 32bit tahr working on my ACER including copying across my saveFile from this laptop. I had not been able to get that working before your patched initrd.gz came along.

Unfortunately that still leaves me in a quandary because the 32bit version can only run video effectively in a very small window. And one of the attractions of the ACER is its bigger and brighter screen.

I am trying to avoid the hassle of installing 64-bit versions fo everything when I already have perfectly good 32 bit programs.

But that problem is not relevant to this Thread.

Unfortunately only Qirky will play videos properly - I would prefer not to need it - I don't like the fact that Quirky does not use a saveFile. And some Java stuff I have does not work properly with it.

...R

[jlst]

I finally applied your patches as the init file i'm using has been slightly edited by me.

The important thing is that it boots to desktop, i'll test the other changes later.

Looking at your changes, I see you missed the FDRV... so here I present you
the code and its areplacement for you to review..

this

Code: Select all

   if [ "$FDRV" = "" -a "$EXTRAFW" = "yes" ];then
    FND_FILES="`find /mnt/data${PSUBDIR} -maxdepth ${SEARCHDEPTH} -xdev -type f -iname ${FDRVSFS} | grep -v ' ' | sed -e 's%^/mnt/data%%' | tr '\n' ' '`"
    for ONEPUPFILE in $FND_FILES
    do
     if [ "$NAMETYPE" = "traditional" ];then
      ONEIDSTRING="$IDSTRING" #found file based on it's name only.
     else
      ONEFULLSIZE=`stat -c %s /mnt/data${ONEPUPFILE}`
      ONEORIGSIZE=`expr $ONEFULLSIZE - 32` #110422
      ONEIDSTRING="`dd if=/mnt/data${ONEPUPFILE} bs=1 skip=${ONEORIGSIZE} 2>/dev/null | sed -e 's%[^a-zA-Z0-9\.]%%g'`"
     fi
     if [ "$ONEIDSTRING" = "$IDSTRING" ];then
      FDRV="${ONEDEV},${ONEFS},${ONEPUPFILE}"
      echo -n " fdrive" > /dev/console
      break
     fi
    done
   fi

becomes:

Code: Select all

   if [ "$FDRV" = "" -a "$EXTRAFW" = "yes" ];then
     find_onepupdrv "$FDRVSPEC" "f" "$FDRVDEV"
     [ "$ONEPUPDRV" ] && { FDRV="$ONEPUPDRV"; FDRVSPEC=""; }
     [ "$FDRV" = "" ] && { find_onepupdrv "$FDRVSFS" "f"; FDRV="$ONEPUPDRV"; }
   fi

=================================================
=================================================

this

Code: Select all

if [ "$FDRV" -a "$EXTRAFW" = "yes" ];then
 DEV="`echo "$FDRV" | cut -f 1 -d ':'`"
 FS="`echo "$PCPARTS0" | grep "${DEV}|" | cut -f 2 -d '|'`"
 SPEC="`echo -n "$FDRV" | cut -f 2 -d ':'`"
 FDRVSFS="`basename $SPEC`"
 FDRV="${DEV},${FS},${SPEC}"
fi

becomes

Code: Select all

[ "$FDRV" -a "$EXTRAFW" = "yes" ] && { decode_spec "$FDRV"; FDRVID="$DID"; FDRVSPEC="$SPEC"; FDRV=""; }

=================================================
=================================================

this

Code: Select all

FLAYER=''
FFACTOR=''
if [ "$FDRV" != "" ];then
  FDEV="`echo "$FDRV" | cut -f 1 -d ','`"
  FFS="`echo "$FDRV" | cut -f 2 -d ','`"
  FFILE="`echo "$FDRV" | cut -f 3 -d ','`"
  MNT_FFILE=""
  fPATTERN="/dev/$FDEV "
  MNT_FFILE="`mount | grep -m1 "$fPATTERN" | cut -f 3 -d ' '`"
  FBASENAME="`basename $FFILE`" #v426 moved up.
  if [ "$MNT_FFILE" != "" ];then
   if [ "$COPY2RAM" = "yes" ];then
    SIZEZK=`du -k ${MNT_FFILE}${FFILE} | cut -f 1`
    SIZEZK=`expr $SIZEZK + 1000` #some slack.
    mount -t tmpfs -o size=${SIZEZK}k tmpfs /mnt/tmpfs5
    cp -af ${MNT_FFILE}${FFILE} /mnt/tmpfs5/
    sync
    losetup /dev/loop5 /mnt/tmpfs5/${FBASENAME}
   else
    losetup /dev/loop5 ${MNT_FFILE}${FFILE}
   fi
   mount -r -t squashfs -o noatime /dev/loop5 /pup_f > /dev/console 2>&1
   if [ $? -eq 0 ];then
    FLAYER=':/pup_f=ro'
    FFACTOR="$FBASENAME" #v426
   fi
  fi
fi

becomes

Code: Select all

FLAYER=''
FFACTOR=''
if [ "$FDRV" != "" ];then
 load_onepupdrv "$FDRV" "f" "5"
 [ "$ONEDRVLAYER" ] && FLAYER="${ONEDRVLAYER}:"
 FFACTOR="$ONEDRVFACTOR"
fi

[gyro]

I finally applied your patches as the init file i'm using has been slightly edited by me.

The important thing is that it boots to desktop, i'll test the other changes later.

Looking at your changes, I see you missed the FDRV... so here I present you
the code and its areplacement for you to review..

Thanks for the code I'll look at including it in patching any "init" that already includes FDRV.
I didn't forget to do FDRV, the project has all been done on "init" code that does not include FDRV. (My fork of /woof-CE/woof-code/huge_extras/init and of course Tahr 6.0.5 and Tahr64 6.0.5.)

Also, thanks for testing.

gyro

[gyro]

@jlst,
I see the problem, my fork of Woof-CE is pre fdrv.
I'll do a new fork and redo the patches with the help of your code. Might produce a single extra patch for all of fdrv stuff, if the current patches still apply.

Folks, please consider the current implementation as proof of concept.
The tahr and tahr64 "initrd.gz" files should still work.

gyro

[gyro]

@Robin2,
I have replied here http://www.murga-linux.com/puppy/viewtopic.php?t=106953
gyro

[gyro]

@jlst,

On a quick review of your code, I think

Code: Select all

 [ "$ONEDRVLAYER" ] && FLAYER="${ONEDRVLAYER}:"

should be

Code: Select all

 [ "$ONEDRVLAYER" ] && FLAYER=":${ONEDRVLAYER}"

Should be like the ZLAYER not the YLAYER.

I now have a current "/woof-CE/woof-code/huge_extras/init" which does include fdrv support.

gyro

[gyro]

@jlist,

There's just one more thing.
To preserve the functionality as per the other drvs

Code: Select all

   if [ "$FDRV" = "" -a "$EXTRAFW" = "yes" ];then
     find_onepupdrv "$FDRVSPEC" "f" "$FDRVDEV"
     [ "$ONEPUPDRV" ] && { FDRV="$ONEPUPDRV"; FDRVSPEC=""; }
     [ "$FDRV" = "" ] && { find_onepupdrv "$FDRVSFS" "f"; FDRV="$ONEPUPDRV"; }
   fi

should be

Code: Select all

   if [ "$EXTRAFW" = "yes" ];then
     find_onepupdrv "$FDRVSPEC" "f" "$FDRVDEV"
     [ "$ONEPUPDRV" ] && { FDRV="$ONEPUPDRV"; FDRVSPEC=""; }
     [ "$FDRV" = "" ] && { find_onepupdrv "$FDRVSFS" "f"; FDRV="$ONEPUPDRV"; }
   fi

A drv specified as a boot parameter should always take precedence over a standard drv.
So, even if "init" has found a standard drv on sdb2, it will still try to use the one specified with a boot parameter on sdb4.
So "find_onepupdrv" is unconditionaly called to look for a boot param drv.

gyro

[Robin2]

Thank you @gyro.

In the meantime I have discovered that there is a XenialPup and (very frustrating) that it existed before I ever started trying to get my ACER working. I probably could have avoided several days of fruitless experiments.

Anyway - that brings me to a slightly different question. Can I apply your patches to the initrd.gz file for XenialPup and how do I do that?

Update
Actually it now seems to be able to find the save file on mmcblk0p1 so I don't need to patch the initrd.gz.

But I still have not figured how to make it boot from the SSD without needing a USB stick.

Many thanks.

...R

[jlst]

Ok, I'll wait for the single patch to apply it again

But maybe these changes are worth adding to the init in the main woofce repo:

- copy itself to new_dest/initrd
- remove umntfunc() and replace it with umount

Those are the changes I added to my init file, so the init stays in sync

[gyro]

All download patches have been replaced with ones that include support for fdrv.
These are now based on "woof-CE/woof-code/huge_extras/init" testing branch, as of 25 May 2016.
I have also attached a copy of that init script with all the patches applied. It's called "init.woof2".

jlist, sorry, in the end I didn't produce a single big patch.

Please see the first post for all downloads.

gyro