Another re-write of the "init" script - using OverlayFs?

Under development: PCMCIA, wireless, etc.
Post Reply
Message
Author
User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#16 Post by peebee »

[FIXED] - see below

Tried both:

Code: Select all

kernel /devtest/vmlinuz  pmedia=atahd pdrv=sda3:/devtest/ psubdir=devtest pfix=fsck
and

Code: Select all

kernel /devtest/vmlinuz  pmedia=atahd pdrv=Linux-2:/devtest/ psubdir=devtest pfix=fsck
In both cases got:
*** Puppy install partition not specified

Code: Select all

# blkid
/dev/sda3: LABEL="Linux-2" UUID="95a633e7-6fab-431d-9fa8-550812ab03f8" TYPE="ext4" PARTUUID="ee50f428-03"
The frugal install folder is /devtest and it's on /dev/sda3.....

What am I missing???
Last edited by peebee on Fri 09 Jun 2017, 07:12, edited 1 time in total.
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#17 Post by gyro »

@peebee,
try

Code: Select all

kernel /devtest/vmlinuz  pmedia=atahd pdrv=sda3:/devtest/ pfix=fsck
or

Code: Select all

kernel /devtest/vmlinuz  pmedia=atahd pdrv=Linux-2 psubdir=/devtest pfix=fsck
My current kernel line is

Code: Select all

  kernel /puppy/slacko/vmlinuz libata.noacpi=1 pmedia=atahd pdrv=Work psubdir=/puppy/slacko pfix=fsckp,nosave,copy
Unfortunately I commented out the line of code that prepends a "/" to psubdir so if specifying a "psubdir=" parameter it must start with a "/". (I'll fix that in the next version.)
Also best tot specify either a directory in the "pdrv=" parameter or with a "psubdir=" paramater, not both.
But I don't understand why "pdrv=sda3:/devtest/" didn't work.
I'll do some testing with boot paramaters like yours, and see if I can get it to fail.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#18 Post by gyro »

The missing "/" on psubdir does cause havoc, only for me the kernel line

Code: Select all

  kernel /puppy/slacko/vmlinuz libata.noacpi=1 pmedia=atahd pdrv=sdb2:/puppy/slacko/ psubdir=puppy/slacko pfix=fsck
Still boots, but it loads only the puppy...sfs file, and hangs on shutdown.
But

Code: Select all

  kernel /puppy/slacko/vmlinuz libata.noacpi=1 pmedia=atahd pdrv=sdb2 psubdir=/puppy/slacko pfix=fsck
worked fine.
I have uploaded a new version that does prepend a "/" to psubdir if it's not specified.
It also definitely does propagate a directory specified as "pdrv=sdb6:/subdir/" as though "psubdir=/subdir" had been specified, provided that "psubdir=" is not specified.
So, these are the equivalent

Code: Select all

pdrv=Work:/subdir/
and

Code: Select all

pdrv=Work psubdir=/subdir
The new version does other stuff as well but I'll cover that in a later post.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

Announcing a new version

#19 Post by gyro »

Thanks to the folk who have tired to run this thing.
Trying to reproduce the problems that have been reported, has led to the fixing of some bugs.
Frustratingly, I still haven't been able to reproduce the same experiences.

But this post is to announce that a new version has been uploaded. (See first post for download utl's.)
Given BarryK's success with an overlay with a simple directory on disk as an rw layer,
I've decided to change the script in two ways.
1. It's now capable of doing the equivalent of a pupmode=5 boot using a directory on a disk as the rw layer instead of a directory in a tmpfs.
If you specify "pfix=nocopy", the sfs's are mounted directly off the disk and a save-directory is created and used as the rw layer.
As a consequence of this, on reboot the save-directory is found and it does a pupmode=12 boot.
This provides persistency without resort to "shutdowconfig". (Of couorse only on a Linux partition.)
In my testing I have a "shutdownconfig" that does nothing if "pfix=nosave" is specified, so I don't have to click on a "No" button on first shutdown.
2. The algorithm has been slightly modified, so that when processing each sfs one by one, each successfully mounted sfs is added to a list, not added to the stack.
Then there is a block of code that builds the stack using these lists of directories.
The idea is that when replacing aufs with overlays, it's all happening in a single block of code.

gyro

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#20 Post by peebee »

[FIXED] see below

Hi gyro - latest initrd.gz.....

I get:
*** Puppy install partition not specified

whatever combination of pdrv and psubdir I try.....

Is it because I'm using uuid??

Last attempt was:

Code: Select all

title Devtest (sda3/devtest)
  uuid 95a633e7-6fab-431d-9fa8-550812ab03f8
  kernel /devtest/vmlinuz  pmedia=atahd pdrv=sda3:/devtest/ pfix=fsck
  initrd /devtest/initrd.gz
Last edited by peebee on Fri 09 Jun 2017, 07:11, edited 1 time in total.
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

User avatar
Keef
Posts: 987
Joined: Thu 20 Dec 2007, 22:12
Location: Staffordshire

#21 Post by Keef »

@peebee

I used UUID and it works for me.

Code: Select all

title tahrpup
  uuid ba259b38-29a0-40e7-84bc-032f1d46204b
  kernel /tahr/vmlinuz psubdir=tahr pmedia=atahd  pdrv=sda2:/tahr/ zdrv=sda2:/tahr/zdrv_tahr_6.0.6.sfs
  initrd /tahr/initrd.gz

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#22 Post by peebee »

Sorry - found my error - I'd put the wrong "version" of DISTRO_SPECS into the frugal subdirectory - I'd used the version which goes into the woof-ce process rather than the output of woof-ce and therefore does not include the *drv specifications.....

With the correct DISTRO_SPECS, boots as expected!
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#23 Post by gyro »

@Keef,
You shouldn't need the "zdrv=" parameter.
I'm successfully booting xenial 7.0.8.1 with this boot entry:

Code: Select all

title Puppy xenial 7.0.8.1 (sdc2/xenial)
  uuid e9e64b47-ccd4-42c5-aeff-e771e264af3c
  kernel /puppy/xenial/vmlinuz acpi_osi=Linux libata.noacpi=1 intel_pstate=disable pdrv=isoSave:/puppy/xenial/ pfix=nosave
  initrd /puppy/xenial/initrd.gz
And it picks up my ydrv and my zdrv.

@peebee,
Great, it now works for you.

Note: I using xenial 7.0.8.1 as my testbed because it has overlays enabled in it's 4.9.13 kernel. And now I'm playing with overlays.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

Announcing a new version

#24 Post by gyro »

I've done it.
The new version I've just uploaded to http://www.fishprogs.software/puppy/initrd/init and http://www.fishprogs.software/puppy/initrd/initrd.gz uses overlays to create the stack instead of aufs.

I used xenial 7.0.8.1 as my test bed, since it's kernel has overlay support as a module.

A normal first boot will boot like a normal pupmode=5 with sfs's copied to ram and the rw layer in tmpfs.
It you add a "pfix=nocopy" boot parameter and the frugal install directory is on a linux partition, it will create and use a savefolder for rw. On second boot it will do a pupmode=12 type boot with the savefolder as the rw layer.

Note: If you see the normal first shutdown config, always click on the "No" button.

Of course this is just first step, a proof of concept, there are lot's of things it can't do.
And a significant amount of effort to sortout exactly what it can do.

Observations:
It seems rather slow at shutting down.
I wonder how much of the rest of puppy has difficulties when booted with this init because aufs is assumed?

gyro
Last edited by gyro on Fri 09 Jun 2017, 19:00, edited 1 time in total.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#25 Post by gyro »

I've just uploaded a couple of files that I find useful with this init.

http://www.fishprogs.software/puppy/ini ... downconfig is a patched "/ust/sbin/shutdownconfig" that honours "pfix=nosave" by doing absolutely nothing.

http://www.fishprogs.software/puppy/initrd/TIME_ZONE is an example file that enables "init" to set the time zone for itself and for the booted puppy. It has to be "/TIME_ZONE" in the "initrd.gz".
The TZ part before the ':' sets the time for "init", and the part after the ':' sets the time for the booted puppy.
Can completely remove all "Time of last mount...." error messages from fsck.

gyro

User avatar
peebee
Posts: 4370
Joined: Sun 21 Sep 2008, 12:31
Location: Worcestershire, UK
Contact:

#26 Post by peebee »

Just to report that the initrd.gz of 9-jun used together with the xenial-7.0.8.1 kernel 4.9.13 booted fine for me.....
also renamed my chromium .sfs to an adrv and that was loaded correctly.
Cheers
peebee
ImageLxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#27 Post by gyro »

@peebee,
I've finally worked out why your deficient DISTRO_SPECS resulted in the misleading error message it did.
The function that deciphers the "pdrv=" boot parameter, does nothing if there's no default filename provided, thus ignoring any valid information that might be provided in the "pdrv=" parameter.
I'll try to fix this in the next version.

gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#28 Post by gyro »

Thanks to all the folk who have taken the time to download and try this stuff, I do appreciate it.
gyro

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

Announcing a new version

#29 Post by gyro »

I've uploaded new versions to http://www.fishprogs.software/puppy/initrd/init and http://www.fishprogs.software/puppy/initrd/initrd.gz.

Changes in this 11 June version:
1. Improved error reporting, particularly in the early "sorting out boot parameters" phase.
2. Interpretation of "pdrv=sdb2:dir/filename.sfs" has changed from "sdb2:/dir/filename.sfs" to "sdb2:/psubdir/dir/filename"
3. Checks for the existence of the "overlay" module, and error exits if it's not found.
4. Writes a "POVERLAY='yes'" line to PUPSTATE, if overlay support is found.
5. Builds the stack in 2 phases, first an ro sfss stack, mounted on "/pup_sfss" and then the full rw stack at "/pup_new".

I've also uploaded a couple of scripts I use a lot when working with "initrd.gz":
http://www.fishprogs.software/puppy/initrd/file2initrd and http://www.fishprogs.software/puppy/initrd/initrd2file.
Just execute them without any parameters to see the "Usage" info.

gyro

User avatar
Moose On The Loose
Posts: 965
Joined: Thu 24 Feb 2011, 14:54

Re: Announcing a new version

#30 Post by Moose On The Loose »

gyro wrote:I've uploaded new versions to http://www.fishprogs.software/puppy/initrd/init and http://www.fishprogs.software/puppy/initrd/initrd.gz.

Changes in this 11 June version:
1. Improved error reporting, particularly in the early "sorting out boot parameters" phase.
2. Interpretation of "pdrv=sdb2:dir/filename.sfs" has changed from "sdb2:/dir/filename.sfs" to "sdb2:/psubdir/dir/filename"
3. Checks for the existence of the "overlay" module, and error exits if it's not found.
4. Writes a "POVERLAY='yes'" line to PUPSTATE, if overlay support is found.
5. Builds the stack in 2 phases, first an ro sfss stack, mounted on "/pup_sfss" and then the full rw stack at "/pup_new".

I've also uploaded a couple of scripts I use a lot when working with "initrd.gz":
http://www.fishprogs.software/puppy/initrd/file2initrd and http://www.fishprogs.software/puppy/initrd/initrd2file.
Just execute them without any parameters to see the "Usage" info.

gyro
I haven't been watching closely but had an idea
Has the loading of an SFS on the fly been made possible yet?
If not perhaps a "fast reboot" could be added that doesn't reboot but instead pivots off the overlayed file system makes changes and goes back onto the modified overlayed system. This would be a lot faster than a complete reboot.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

Re: Announcing a new version

#31 Post by gyro »

Moose On The Loose wrote:Has the loading of an SFS on the fly been made possible yet?
No, I have still not found a way to modify an existing overlay stack, but testing of this is not yet complete.
Moose On The Loose wrote:If not perhaps a "fast reboot" could be added that doesn't reboot but instead pivots off the overlayed file system makes changes and goes back onto the modified overlayed system. This would be a lot faster than a complete reboot.
Or could try building a new "fixed" stack and pivot root to that.

gyro

Rangan Masti
Posts: 37
Joined: Tue 01 Jan 2013, 19:23
Location: Germany, Berlin

#32 Post by Rangan Masti »

Wish you success guys.

User avatar
Moose On The Loose
Posts: 965
Joined: Thu 24 Feb 2011, 14:54

Re: Announcing a new version

#33 Post by Moose On The Loose »

gyro wrote:
Moose On The Loose wrote:Has the loading of an SFS on the fly been made possible yet?
No, I have still not found a way to modify an existing overlay stack, but testing of this is not yet complete.
Moose On The Loose wrote:If not perhaps a "fast reboot" could be added that doesn't reboot but instead pivots off the overlayed file system makes changes and goes back onto the modified overlayed system. This would be a lot faster than a complete reboot.
Or could try building a new "fixed" stack and pivot root to that.

gyro
Yes, if a modified stack can be done, it seems like a faster way to get to a changed system. It may take some trickery to make the changes up to that point stay but not override the new SFS.

gyro
Posts: 1798
Joined: Tue 28 Oct 2008, 21:35
Location: Brisbane, Australia

#34 Post by gyro »

When I tried to change the list of ro directories in an overlay with a "remount", there were no error messages, but the overlay remained unchanged. So this seems to be a dead-end.

But I was able to create a new overlay using some directories from the old overlay (including the rw dir, "upperdir" and "workdir") + a new ro directory. Then unmounting the old overlay and moving the new overlay to the original mount point. This worked.
Although this was just using play directories, not the actual / overlay.
So there is still some hope of being able to create a new overlay at /pup_new and doing some sort of pivot root to /pup_new, (again).

Unfortunately, it is probably safer to start by returning to the ancient puppy method of building the whole stack in "init" and not modifying it at all.

I've uploaded a new version of the "init" and "initrd.gz".
This one creates an ro overlay containing just the sfs's at /pup_sfss, and after sorting out what directory to use as an rw directory, creates the full overlay on /pup_new, using /pup_sfss as the "lowerdir".
This restores the logic of dealing with the sfs's first, then sorting out the save layer.
This also means that in the running system /initrd/pup_sfss contains the files of all the sfs's.

This version is also the first one that writes a PUPSTATE where the variables that are written are fully compatible with current puppies.

gyro

Rangan Masti
Posts: 37
Joined: Tue 01 Jan 2013, 19:23
Location: Germany, Berlin

#35 Post by Rangan Masti »

Hi Good morning, where to find the new initrd uploaded.for testing?
Thank you and have a great day!

Post Reply