Another re-write of the "init" script - using OverlayFs?
-
- Posts: 37
- Joined: Tue 01 Jan 2013, 19:23
- Location: Germany, Berlin
overlay init 0.1 instructions from 07/01/18
Hi, good afternoon. I am using the new initrd on both Slacko64 and Slacko32.
Working fine for me sofar!. Tahnk you. Have a nice day.
Rangan.
Working fine for me sofar!. Tahnk you. Have a nice day.
Rangan.
Re: overlay init 0.1 instructions from 07/01/18
Thanks for testing and reportingRangan Masti wrote:I am using the new initrd on both Slacko64 and Slacko32. Working fine for me sofar!.
gyro
The idea of a delta to an iso was to make it easier to try.01micko wrote:IMHO I don't think a delta is the way to go if the default is to check the filesystem integrity and time skew errors are reported.
I need to create a patched .iso to be able to test CD booting, and installers etc. So I thought I might publish a delta of this iso.
No, for most reliable results, fsck should be run before the first mount on every boot, i.e. as this init does.01micko wrote:While it is probably important to check for filesystem errors, couldn't this be done in OS proper? Before a save is created or at least after TZ is set by the user?
So idealy the time skew issue needs to be solved in init before the first mount.
If a TIME_ZONE file is present in the initrd.gz, this is exactly what this init script does.
The problem is that the current TIME_ZONE stuff works fine in this test setup environment, where the overlay-setup script can add a TIME_ZONE file to the initrd.gz.
But it won't if there is a published iso, which people expect to just boot directly or install with a standard installer. Since no published iso should contain a TIME_ZONE file.
I have no simple solution at this time.
Theoretically we could modify all the Puppy installers to generate a TIME_ZONE file from the current Puppy and add it to the installed initrd.gz.
(Modifying and maintaining all the Puppy installers is the hard thing here.)
This would not do anything for booting from a CD or iso.
Or the running system could signal the selected TZ to the init script.
This does not resolve the issue for first boot, but does for subsequent boots.
This might be the best compromise.
This TIME_ZONE issue is one of the things I'm working on for version 0.2.
@01micko, thanks for testing, and for the pdf.
gyro
This usually indicates your computers internal hardware clock is not working correctly and does not keep correct time.The TIME_ZONE file:
It's important function is to set the TZ variable, hence the local time-zone in "init", before any partitions are mounted. Thus eliminating any complaints from fsck relatiing to times being in the future.
Since, by default, this "init" does an fsck before the first mount of any partition, I became annoyed with frequently seeing these messages.
This is one way to get around this problem.
fake-hwclock
http://manpages.ubuntu.com/manpages/tru ... ock.8.html
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
Sorry, I have to disagree.bigpup wrote:This usually indicates your computers internal hardware clock is not working correctly and does not keep correct time.
The hardware clock is fine, the problem is that the 'init' script does not know what the actual timezone is, while it's busy mounting and unmounting partitions.
The Puppy tradidional way, is for 'init' to set the timezone to some hardcoded value (hence usually wrong). The problem wih this is that the superblock times in a Linux partition can get skewed into the future either when you reboot to this Puppy or when you reboot to another Puppy, depending on which side of your actual timezone the hardcoded timezone is set to.
The concept of the TIME_ZONE file is to allow 'init' to correctly set the timezone, before any mounts or unmounts of partitions.
The good news about all this timezone stuff is that it does not affect the integrity of the data. It's just annoying messages from fsck.
gyro
What about for ISO booting purposes, hard coding to Pacific/Kiritimati? That is the eastern most time zone so therefore anywhere after must be in the past. I notice fsck doesn't care if the time is in the past. I just checked by skipping my clock back an hour (future superblock error is reported) and skipping it forward an hour and no error reported. Of course I have Australia/Queensland set in my TIME_ZONE file.gyro wrote:The problem wih this is that the superblock times in a Linux partition can get skewed into the future either when you reboot to this Puppy or when you reboot to another Puppy, depending on which side of your actual timezone the hardcoded timezone is set to.
In other news, I did the deed with a fresh slacko64 install with kernel updated to 4.9.73 with necessary configs. I approached it a bit differently though. I did the fresh frugal and then booted it, set timezone and from within that install performed all of your modifications. Seems to be working quite well. I have a google-chrome sfs loaded running as 'spot' with added support packages added from PPM (gtk+3, updated mozilla-nss etc).
I'll keep this as a daily driver for a while.
Thanks!
Puppy Linux Blog - contact me for access
Tested with:
LxPupSc-18.01+4T-k64
Kernel 4.14.13 64-bit
All seems fine
Cheers
peebee
menu.lst=
LxPupSc-18.01+4T-k64
Kernel 4.14.13 64-bit
All seems fine
Cheers
peebee
menu.lst=
Code: Select all
title OverlayFS (sda4/overlayfs)
uuid 9bb32631-a1f9-425f-81bb-9f816347e8fc
kernel /overlayfs/vmlinuz pmedia=atahd pdrv=sda4 psubdir=overlayfs
initrd /overlayfs/initrd.gz
Code: Select all
====> /initrd/tmp/bootinit.log <====
'FATAL' messages may be insignificant.
Doing fsck of /dev/sda4 as ext4 with e2fsck.
e2fsck 1.43.3 (04-Sep-2016)
Linux-3: clean, 303495/5693440 files, 15090214/22761984 blocks
Mounting /dev/sda4 on /mnt/sda4 as ext4.
Using /mnt/sda4/overlayfs/BOOT_SPECS
Using 'puppy_LxPupSc_18.01.sfs' as sfs0 file...
Copying /mnt/sda4/overlayfs/puppy_LxPupSc_18.01.sfs to /mnt/tmpfs/
Mounting /mnt/tmpfs/puppy_LxPupSc_18.01.sfs on /pup_sfs0.
Using 'fdrv_LxPupSc_18.01.sfs' as sfs1 file...
Copying /mnt/sda4/overlayfs/fdrv_LxPupSc_18.01.sfs to /mnt/tmpfs/
Mounting /mnt/tmpfs/fdrv_LxPupSc_18.01.sfs on /pup_sfs1.
Using 'zdrv_LxPupSc_18.01.sfs' as sfs2 file...
Copying /mnt/sda4/overlayfs/zdrv_LxPupSc_18.01.sfs to /mnt/tmpfs/
Mounting /mnt/tmpfs/zdrv_LxPupSc_18.01.sfs on /pup_sfs2.
Using 'overlay_mods.sfs' as sfs3 file...
Copying /mnt/sda4/overlayfs/overlay_mods.sfs to /mnt/tmpfs/
Mounting /mnt/tmpfs/overlay_mods.sfs on /pup_sfs3.
Using 'adrv_LxPupSc_18.01.sfs' as sfs4 file...
Copying /mnt/sda4/overlayfs/adrv_LxPupSc_18.01.sfs to /mnt/tmpfs/
Mounting /mnt/tmpfs/adrv_LxPupSc_18.01.sfs on /pup_sfs4.
Adding module kernel/fs/overlayfs/overlay.ko
Mounting pup_sfs3:pup_sfs4:pup_sfs0:pup_sfs1:pup_sfs2 on /pup_sfs...
umount_unneeded - keeping /dev/sda4
Booting with PUPMODE=12
Using personal storage /overlayfs/LxPupScsave [sda4].
Mounting LxPupScsave:pup_sfs on /pup_new...
mount -o move /pup_sfs0 /pup_new/initrd/pup_sfs0
mount -o move /pup_sfs1 /pup_new/initrd/pup_sfs1
mount -o move /pup_sfs2 /pup_new/initrd/pup_sfs2
mount -o move /pup_sfs3 /pup_new/initrd/pup_sfs3
mount -o move /pup_sfs4 /pup_new/initrd/pup_sfs4
mount -o move /pup_sfs /pup_new/initrd/pup_sfs
mount -o move /mnt/sda4 /pup_new/initrd/mnt/sda4
mount -o move /mnt/tmpfs /pup_new/initrd/mnt/tmpfs
'/pup_new/initrd/pup_rw' -> '/initrd/mnt/sda4/overlayfs/LxPupScsave'
'/pup_new/tmp' -> '/initrd/mnt/tmpfs/tmp'
- Attachments
-
- Screenshot.png
- (179.07 KiB) Downloaded 471 times
Last edited by peebee on Sun 14 Jan 2018, 09:36, edited 1 time in total.
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
I got interested when I read this, so I've done a few tests.01micko wrote:gyro wrote:The problem wih this is that the superblock times in a Linux partition can get skewed into the future either when you reboot to this Puppy or when you reboot to another Puppy, depending on which side of your actual timezone the hardcoded timezone is set to.
I found out that this superblock problem only manifest itself when your hardware clock is set to localtime - which is the default settings in most puppies, to accomodate dual booting with Windows. If hwclock is set in UTC, all is good.
I don't know how you do it gyro, but this is how I did it in init (I don't have persistence file in initrd like you do): "[ $TZ ] && hwclock -t" where hwclock is busybox applet similar to coreutils' utility of the same name.
This way, I only need to pass boot parameter TZ=EST-10 and all will be right. Of course, again, this is only necessary when running hwclock in localtime.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]
I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso
See: http://murga-linux.com/puppy/viewtopic. ... 329#980329
See: http://murga-linux.com/puppy/viewtopic. ... 329#980329
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
I tried this with artful pup. opened the initrd and put in the DISTRO_SPECS. added kernel arguments pdrv and psubdir and everything else was automatic.
Once at desktop loaded wine-2.2.sfs @200mb. the 32bit pae kernel recognised 7.9gb of 16gb.
Once at desktop loaded wine-2.2.sfs @200mb. the 32bit pae kernel recognised 7.9gb of 16gb.
- Attachments
-
- Screenshot(3).png
- (117.18 KiB) Downloaded 157 times
Thanks.peebee wrote:I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso
See: http://murga-linux.com/puppy/viewtopic. ... 329#980329
I assume you did not inclued a TIME_ZONE in the initrd.gz in this iso.
gyro
Yes I did - 01Micko's suggestion as mentioned in the link....gyro wrote:Thanks.peebee wrote:I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso
See: http://murga-linux.com/puppy/viewtopic. ... 329#980329
I assume you did not inclued a TIME_ZONE in the initrd.gz in this iso.
gyro
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Firstly, thanks folk for taking the time to test this.
Re the timezone thing:
1. Skewing only occurs if the hardware clock is set to local time.
2. If you set it to a fixed value you see the skew reported either:
When you mount a partition with the correct timezone that was umounted with the fixed timezone,
or
When you mount a partition with the fixed timezone that was umounted with the corrent timezone.
3. This init seeks to provide facilities so that the timezone can set consistently for all mounts.
4. The TIME_ZONE file idea makes most sense when you are modifying the initrd.gz before first boot, it does not make as much sense when this intrd.gz is already included in an iso.
5. Version 0.2 will include support for a "ptz" boot parameter. e.g. ptz=AEST-10
6. Version 0.2 will also support the possibility of the time-zone in init being set by some utility in the running system, possibly 'Quick Setup'. The skew issue could still occur on first boot.
gyro
Re the timezone thing:
1. Skewing only occurs if the hardware clock is set to local time.
2. If you set it to a fixed value you see the skew reported either:
When you mount a partition with the correct timezone that was umounted with the fixed timezone,
or
When you mount a partition with the fixed timezone that was umounted with the corrent timezone.
3. This init seeks to provide facilities so that the timezone can set consistently for all mounts.
4. The TIME_ZONE file idea makes most sense when you are modifying the initrd.gz before first boot, it does not make as much sense when this intrd.gz is already included in an iso.
5. Version 0.2 will include support for a "ptz" boot parameter. e.g. ptz=AEST-10
6. Version 0.2 will also support the possibility of the time-zone in init being set by some utility in the running system, possibly 'Quick Setup'. The skew issue could still occur on first boot.
gyro
Sorry, but I don't think it's a good idea to use the timzone setting facilities in this manner. If you are going to set the timezone it should be to the correct one.peebee wrote:Yes I did - 01Micko's suggestion as mentioned in the link....gyro wrote:Thanks.peebee wrote:I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso
See: http://murga-linux.com/puppy/viewtopic. ... 329#980329
I assume you did not inclued a TIME_ZONE in the initrd.gz in this iso.
gyro
Which is one of the problems when trying to use version 0.1 in an iso.
Unfortunately version 0.1 effectively sets the timezone to UTC in the absense of a TIME_ZONE file.
Version 0.2 will set it to 'XXX12' (west of everwhere), if it is not set in a TIME_ZONE file or ptz boot parameter.
(I need to test the west v east thing again before releasing version 0.2)
gyro
There is no "correct" one for a general purpose .iso.....gyro wrote:If you are going to set the timezone it should be to the correct one.
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Yes, that's what I was trying to say.peebee wrote:There is no "correct" one for a general purpose .iso.....gyro wrote:If you are going to set the timezone it should be to the correct one.
The 'init' needs to have a hardcoded default that ensures that no one sees the issue on first fsck on first boot.
Then provide mechanisms for the correct local timezone to be set in 'init'.
This is what I have coded into "overlay_init-0.2". (Just a bit more testing to do)
Then there is no point in having a TIME_ZONE file in the iso.
gyro
I downloaded this and used it for a fresh manual frugal install.peebee wrote:I've made a delta to produce LxPupSc-18.01+4-overlayfs.iso
See: http://murga-linux.com/puppy/viewtopic. ... 329#980329
It appeared to work fine.
I then extracted the TIME_ZONE file from the "initrd.gz" and realised that it appeared to work because it didn't work.
The problem is the TIME_ZONE file:
Code: Select all
<+14>-14|Pacific/Kiritimati
I changed the TIME_ZONE file to:
Code: Select all
LINT-14|Pacific/Kiritimati
When I removed the TIME_ZONE file from "initrd.gz" with "initrdDfile", added "TZ=AEST-10" as a boot parameter, and did another first boot, I got no warnings.
The problem with the TIME_ZONE file is mine, "mk-timezone-file" simply accepted this incompatible format.
This is fixed in the next version which will replace:
Code: Select all
<+14>-14
Code: Select all
XXX-14
gyro
Release of overlay_init-0.2
Release of overlay_init-0.2
Download from http://www.fishprogs.software/puppy/ini ... it-0.2.tar.
Changes:
1. New timezone handling including a new format for the "TIME_ZONE" file, example:So if you have an existing "TIME_ZONE" file you will need to run "mk-timezone-file" again.
The timezone in "init" can be set in any 1 of 3 ways:
Create a "TIME_ZONE" file with "mk-timezone-file" and add it to the "initrd.gz" with "initrd-pup-file".
Add "TZ=AEST-10" as a boot parameter, use "showTZ" to see an appropriate value for your timezone.
Run "set-init-timezone" after the timezone has been set in the running Puppy. The timezone in "init" will then be set for subsequent boots.
In the absence of any specified timezone, "init" uses "TZ=XXX12", the most westerly timezone.
2. All the timezone handling utilities have been fixed to handle the stange format of some TZ specs at the end of some zoneinfo files.
3. New Targeted update for "rc.update", after sfs stack changes.
When changes to the sfs stack are detected, the sfs is checked to see what updates are required, similar to current "sfs_load".
This is done in both the sfs manager utilities, and "rc.update" itself.
Then "rc.update" only performs the appropriate updates.
"BOOTCONFIG" is no longer used, instead "PREVUNIONRECORD" and "LASTUNIONRECORD" are files on their own.
4. Changed the "Archive" save mechanism to use "PUPMODE=37".
5. Renamed some CLI utilities:
bootspecs2tmpbs -> bootspecs-init
tmpbs2bootspecs -> bootspecs-save
tmpbs2variable -> bs-read-var
variable2tmpbs -> bs-write-var
tmpbsDvariable -> bs-delete-var
initrd2file -> initrd-get-file
file2initrd -> initrd-put-file
initrdDfile -> initrd-delete-file
6. Combined "lsoverlay" and "showoverlay", they now both provide the same output.
7. Removed support for "pimod" boot parameter, it's not really needed.
To try this version, use the same procedure as for overlay_init-0.1,
see http://www.murga-linux.com/puppy/viewto ... 6&start=96
Manual update of overlay_init-0.1 install, that might work:
1. Extract the tar file.
2. Replace the current "overlay_mods.sfs".
3. Extract the updated "init" from "initrd.overlay.gz" with "initrd2file" or "initrd-get-file".
4. Store this updated "init" in the currrent "initrd.gz" with "file2initrd" or "initrd-pup_file".
5. If your current "initrd.gz" contains a "TIME_ZONE" file you need to replace it with a new one.
gyro
Download from http://www.fishprogs.software/puppy/ini ... it-0.2.tar.
Changes:
1. New timezone handling including a new format for the "TIME_ZONE" file, example:
Code: Select all
P_TZONE='Australia/Queensland'
P_TZ='AEST-10'
The timezone in "init" can be set in any 1 of 3 ways:
Create a "TIME_ZONE" file with "mk-timezone-file" and add it to the "initrd.gz" with "initrd-pup-file".
Add "TZ=AEST-10" as a boot parameter, use "showTZ" to see an appropriate value for your timezone.
Run "set-init-timezone" after the timezone has been set in the running Puppy. The timezone in "init" will then be set for subsequent boots.
In the absence of any specified timezone, "init" uses "TZ=XXX12", the most westerly timezone.
2. All the timezone handling utilities have been fixed to handle the stange format of some TZ specs at the end of some zoneinfo files.
3. New Targeted update for "rc.update", after sfs stack changes.
When changes to the sfs stack are detected, the sfs is checked to see what updates are required, similar to current "sfs_load".
This is done in both the sfs manager utilities, and "rc.update" itself.
Then "rc.update" only performs the appropriate updates.
"BOOTCONFIG" is no longer used, instead "PREVUNIONRECORD" and "LASTUNIONRECORD" are files on their own.
4. Changed the "Archive" save mechanism to use "PUPMODE=37".
5. Renamed some CLI utilities:
bootspecs2tmpbs -> bootspecs-init
tmpbs2bootspecs -> bootspecs-save
tmpbs2variable -> bs-read-var
variable2tmpbs -> bs-write-var
tmpbsDvariable -> bs-delete-var
initrd2file -> initrd-get-file
file2initrd -> initrd-put-file
initrdDfile -> initrd-delete-file
6. Combined "lsoverlay" and "showoverlay", they now both provide the same output.
7. Removed support for "pimod" boot parameter, it's not really needed.
To try this version, use the same procedure as for overlay_init-0.1,
see http://www.murga-linux.com/puppy/viewto ... 6&start=96
Manual update of overlay_init-0.1 install, that might work:
1. Extract the tar file.
2. Replace the current "overlay_mods.sfs".
3. Extract the updated "init" from "initrd.overlay.gz" with "initrd2file" or "initrd-get-file".
4. Store this updated "init" in the currrent "initrd.gz" with "file2initrd" or "initrd-pup_file".
5. If your current "initrd.gz" contains a "TIME_ZONE" file you need to replace it with a new one.
gyro
New delta for LxPupSc made with overlay_init-0.2....
Timezone not altered during make process - boots with GMT-8....
Timezone not altered during make process - boots with GMT-8....
LxPup = Puppy + LXDE
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Main version used daily: LxPupSc; Assembler of UPups, ScPup & ScPup64, LxPup, LxPupSc & LxPupSc64
Thanks, I'll give it a spin.peebee wrote:New delta for LxPupSc made with overlay_init-0.2....
Timezone not altered during make process - boots with GMT-8....
Later:
It works as expected.
I did a normal manual frugal install from the iso.
Booted, and as expected the timezone was GMT-8.
After doing "Quick Setup", I ran "mk-timezone-file", and added it into the "initrd.gz" with "initrd-put-file".
On first shutdown I chose 'Archive'.
On reboot all was as expected, I did not get a first boot, all the choices from the first "Quick Setup" were still in place.
gyro