Avoid "Searching for Puppy files" during bootup - revisited
Well i never found the mount unmount routine to be significant in time usage.... the use of find and multilevel searches followed by rationalizing the resultant pile of info on the other hand was a big hit ..way slower than the original use of ls <pattern match> .
After all the init goes through a myriad of hoops just to generate the pupsave variable...so why not supply it and save the trip.
mike
ps...I did wonder how grub searches and find files....but probably not something we could use for this ... find... set kernel parameter idea....
After all the init goes through a myriad of hoops just to generate the pupsave variable...so why not supply it and save the trip.
mike
ps...I did wonder how grub searches and find files....but probably not something we could use for this ... find... set kernel parameter idea....
Another little issue is that $pupsave works fine when savefile/folder is in another disk. In this case /mnt/home is a broken link. Adding psavemark is useless in this case.
BTW savefile/folder is in another partition of the boot disk works fine for /mnt/home, with of without psavemark.
Some code should be added so either pupsave should not be allowed in another disk or /mnt/home also consider safefile/fodler drive in addition to partition.
I think the latter is better.
BTW savefile/folder is in another partition of the boot disk works fine for /mnt/home, with of without psavemark.
Some code should be added so either pupsave should not be allowed in another disk or /mnt/home also consider safefile/fodler drive in addition to partition.
I think the latter is better.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==
It makes coding way way easier if one works on the basis that an install is located in one place rather than scattered over several devices...that one grew out of the live CD usage and a lack of comfortable booting options and has stuck even though methods have moved on/improved.
I don't have floppies and only have an external DVD writer now for starters...and my gear is old. A superfloppy zip drive... never even seen one.
Mike
I don't have floppies and only have an external DVD writer now for starters...and my gear is old. A superfloppy zip drive... never even seen one.
Mike
The basic assumption is that the user all (s)he wants is to use their machine with no hassle what so ever and no idea of its inner workings and requirements are.mikeb wrote:It makes coding way way easier if one works on the basis that an install is located in one place rather than scattered over several devices...that one grew out of the live CD usage and a lack of comfortable booting options and has stuck even though methods have moved on/improved.
There are mini-distros that require you to define a number of cryptic, complex and precise options to boot, and this is perfectly fine. Puppy, to keep true to its goals, should rather give the ability of "cryptic, complex and precise boot options" if you want to use them, but still be forgiving and relaxed about them.
BTW the /mnt/home issue is not as bad as originally reported. Is only the first time that has an issue. Then it behaves better. Still needs to be looked at though.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==
hmm i found the puppy 2 approach fulfilled that requirement quite nicely and 4+ meant an increasing number of confusing boot parameters and a complex init script with quirky hbehaviour.. As a noob I found puppy 2 straightforward. Hence the interest in gyros re-evaluation of the boot methods in use....bit of a pet subject ..no pun intended.The basic assumption is that the user all (s)he wants is to use their machine with no hassle what so ever and no idea of its inner workings and requirements are.
Anything to streamline and keep user friendly is welcome.
mike
Gyro's additions are of course welcome, but they do not change anything in the boot process. They do add another option for a kernel command line argument which is very welcome. We just want to make it as much as possible user friendly and false-tolerant.mikeb wrote: Hence the interest in gyros re-evaluation of the boot methods in use....bit of a pet subject ..no pun intended.
Anything to streamline and keep user friendly is welcome.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==
updated patch to init.nosearch-3.diff
Updated patch to "init.nosearch-3.diff" see first post for download.
This is a significant rewrite to avoid extra mount and umount.
The code at the beginning of the "Searching" section does not set the "PUPSAVE" variable, rather it just checks that the partition exists and stores data for later.
The decision is now made in the "search_func" after the first partition is mounted.
The variable "LESSPARTS" is modified so that the specified partition is first in the list.
If the specified savefile/savefolder is not found, then normal searching is performed.
@mavrothal,
"psavemark" is irrelevent since that should have been sorted out before "nosearch" is activated, so it is now recorded in "PUPSTATE" and hence gets respected. The idea behind "nosearch" is that you get it all sorted out running normally, until "PUPSTATE" contains all the appropriate information. Then "nosearch" is activated by specifying a "pupsave" parameter.
I don't quite understand the problem with "/mnt/home".
The "pupsave" parameter could be used to choose between multiple savefiles/savefolders but I don't see any use for specifying the savefile/savefolder of another puppy.
Note: I will probably announce "initrd.gz" files for Tahrpup 6.0.2 in the near future. (My site is up again.)
Edit: now done.
gyro
This is a significant rewrite to avoid extra mount and umount.
The code at the beginning of the "Searching" section does not set the "PUPSAVE" variable, rather it just checks that the partition exists and stores data for later.
The decision is now made in the "search_func" after the first partition is mounted.
The variable "LESSPARTS" is modified so that the specified partition is first in the list.
If the specified savefile/savefolder is not found, then normal searching is performed.
@mavrothal,
"psavemark" is irrelevent since that should have been sorted out before "nosearch" is activated, so it is now recorded in "PUPSTATE" and hence gets respected. The idea behind "nosearch" is that you get it all sorted out running normally, until "PUPSTATE" contains all the appropriate information. Then "nosearch" is activated by specifying a "pupsave" parameter.
I don't quite understand the problem with "/mnt/home".
The "pupsave" parameter could be used to choose between multiple savefiles/savefolders but I don't see any use for specifying the savefile/savefolder of another puppy.
Note: I will probably announce "initrd.gz" files for Tahrpup 6.0.2 in the near future. (My site is up again.)
Edit: now done.
gyro
Last edited by gyro on Tue 30 Jun 2015, 02:58, edited 1 time in total.
Just a thought:
One of the difficulties in implementing "nosearch" by reading "PUPSTATE" from the savefile/savefolder is that the issues of it being a savefile or savefolder, and is the savefile encrypted, have to be sorted before "PUPSTATE" can be read.
It would probably be easier to implement "nosearch" if it were triggered by "pdev1" boot parameter being supplied, and the presence of "${PDEV1}${PSUBDIR}/PUPSTATE".
gyro
One of the difficulties in implementing "nosearch" by reading "PUPSTATE" from the savefile/savefolder is that the issues of it being a savefile or savefolder, and is the savefile encrypted, have to be sorted before "PUPSTATE" can be read.
It would probably be easier to implement "nosearch" if it were triggered by "pdev1" boot parameter being supplied, and the presence of "${PDEV1}${PSUBDIR}/PUPSTATE".
gyro
validating existence of boot parameter sfs files
@mavrothal,
Here is a patch to check the existence of an "adrv" boot parameter file:The idea is to delay checking the file until the "search_func", when a partition is already mounted. A bit like my latest "nosearch" patch.
Obviously the same approach could be applied to the other sfs parameters.
gyro
Here is a patch to check the existence of an "adrv" boot parameter file:
Code: Select all
--- init.orig 2015-06-24 09:52:21.966056508 +1000
+++ init 2015-06-28 19:47:14.014732397 +1000
@@ -473,6 +473,13 @@
done
fi
+ if [ "$PADRVDEV" = "$ONEDEV" ];then
+ if [ -s "/mnt/data${PADRVSPEC}" ];then
+ ADRVSFS="`basename $PADRVSPEC`"
+ ADRV="${ONEDEV},${ONEFS},${PADRVSPEC}"
+ echo -n " adrive" > /dev/console
+ fi
+ fi
if [ "$ADRV" = "" ];then
FND_FILES="`find /mnt/data${PSUBDIR} -maxdepth ${SEARCHDEPTH} -xdev -type f -iname ${ADRVSFS} | grep -v ' ' | sed -e 's%^/mnt/data%%' | tr '\n' ' '`"
for ONEPUPFILE in $FND_FILES
@@ -799,11 +806,13 @@
fi
if [ "$ADRV" ];then
- DEV="`echo "$ADRV" | cut -f 1 -d ':'`"
- FS="`echo "$PCPARTS0" | grep "${DEV}|" | cut -f 2 -d '|'`"
- SPEC="`echo -n "$ADRV" | cut -f 2 -d ':'`"
- ADRVSFS="`basename $SPEC`"
- ADRV="${DEV},${FS},${SPEC}"
+ PADRVDEV="`echo "$ADRV" | cut -f 1 -d ':'`"
+ if [ "`echo "$PCPARTS0" | grep "${PADRVDEV}|"`" != "" ];then
+ PADRVSPEC="`echo -n "$ADRV" | cut -f 2 -d ':'`"
+ else
+ PADRVDEV=""
+ fi
+ ADRV=""
fi
if [ "$YDRV" ];then
Obviously the same approach could be applied to the other sfs parameters.
gyro
Re: validating existence of boot parameter sfs files
gyro wrote:@mavrothal,
Here is a patch to check the existence of an "adrv" boot parameter file:Code: Select all
-[/quote] Looks good. You may want to send a pull request with the changes. If you get ti it pls try to include verifications for $zdrv and $pupsfs as well as. Thx
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==
Re: validating existence of boot parameter sfs files
My plan is to implement a full patch that includes all the sfs parameters, and then announce it in a separate topic to give others a better chance of commenting. After all this topic is about "nosearch". Pull request later.mavrothal wrote:You may want to send a pull request with the changes.
If you get ti it pls try to include verifications for $zdrv and $pupsfs as well as.
I've simplified the code a bit so it is in only 2 places, where the boot parameters are first processed, and in search_func.
In testing, I liked being able to specify an adrv with the wrong name, and in a different directory.
gyro
Re: validating existence of boot parameter sfs files
Why adrv only?gyro wrote:I liked being able to specify an adrv with the wrong name, and in a different directory.
Theoretically SFSs can be anywhere as long as they are loaded in RAM. You do not care after that.
You can actually have a small 10MB partition for vmlinuz/initrd/grub(2) and then point to anywhere you want.
One thing that should be checked with the "anywhere" approach is how the system will behave if SFSs are not loaded in RAM either because there is not enough or because they are called with pfix=nocopy and are in different drives.
== [url=http://www.catb.org/esr/faqs/smart-questions.html]Here is how to solve your[/url] [url=https://www.chiark.greenend.org.uk/~sgtatham/bugs.html]Linux problems fast[/url] ==
Re: validating existence of boot parameter sfs files
Because that is all I had implemented at the time, as a proof of concept.mavrothal wrote:Why adrv only?
The first limitation is which partitions get mounted during the "Loading puppy files" section of "init". See http://www.murga-linux.com/puppy/viewtopic.php?t=100011. Copying to RAM overcomes only the second challenge; which partitions remain mounted.mavrothal wrote:Theoretically SFSs can be anywhere as long as they are loaded in RAM.
Please continue any discussion of sfs boot parameters in the other topic.
This topic is about "nosearch".
gyro
Updated "initrd.gz" files for version 3 available
I have uploaded "initrd.gz" files for Tahrpup 6.0.2 for version 3 of the patch.
See the first post for download details.
gyro
See the first post for download details.
gyro
Tested with and without pupsave kernel parameter in X-tahr-1b2 non-PAE on a core 2 duo laptop and on one of my Bay Trail desktops with the very slow USB flag returns. In these two tests the modified init does just what it is supposed to. Speedup on both systems is the same as my ignore=usb modification to init. If this were to be incorporated in the 'stock' inits, it would save me folding in my patch when using a new initrd.
Thanks,
Edit: Init script also used in X-slacko-3.0n. Booting correctly with and without pupsave parameter there also. All of these tests are on Grub4Dos frugal installs with standard unencrypted savefiles.
Thanks,
Edit: Init script also used in X-slacko-3.0n. Booting correctly with and without pupsave parameter there also. All of these tests are on Grub4Dos frugal installs with standard unencrypted savefiles.
Pups currently in kennel :D Older LxPupSc and X-slacko-4.4 for my users; LxPupSc, LxPupSc64 and upupEF for me. All good pups indeed, and all running savefiles for look'n'feel only. Browsers, etc. solely from SFS.
- technosaurus
- Posts: 4853
- Joined: Mon 19 May 2008, 01:24
- Location: Blue Springs, MO
- Contact:
WOW, I can't believe how overly complex this has become ... when it could just be a single parameter SFS= with a ',' separated list of files to mount and use full paths vs. search if an item in the list starts with '/'
Here is some blah template code example ... mount_sfs omitted for brevity ... needs to be updated for overlayfs anyhow
Here is some blah template code example ... mount_sfs omitted for brevity ... needs to be updated for overlayfs anyhow
Code: Select all
file_missing(){
echo "$1 not found, add the appropriate mount point to MNT="
}
autoload_sfs(){
for MTPT in /mnt/*; do
[ -f "$MTPT/$1" ] && mount_sfs "$MTPT/$1" && break
done
file_missing "$1"
}
load_sfs(){
[ -f "$1" ] && mount_sfs "$1" || file_missing "$1"
}
mount_sfs(){
echo "mounting $1" #etc... and so forth
}
handle_sfs_list(){
while ([ "$1" ]) do
case "$1" in
/*)load_sfs "$1";;
*)[ "$NOSEARCH" ] || autoload_sfs "$1";;
esac
shift
done
}
SFS=a.sfs,/mnt/sda1/b.sfs,...
IFS=","
handle_sfs_list $SFS
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].
Some of the complexity comes from trying to efficiently integrate it into the 'init' script.technosaurus wrote:WOW, I can't believe how overly complex this has become
Are you suggesting that this code in the 'init' scripttechnosaurus wrote:when it could just be a single parameter SFS= with a ',' separated list of files to mount and use full paths vs. search if an item in the list starts with '/'
Code: Select all
#100915 requested by technosaurus (formats get changed further down)...
[ $pupsfs ] && PUPSFS=$pupsfs #format partition:<path><filename> ex: sda2:/wary071/wary_071.sfs
[ $zdrv ] && ZDRV=$zdrv #ex: sda2:/wary071/zdrv_071.sfs
[ $adrv ] && ADRV=$adrv
[ $ydrv ] && YDRV=$ydrv
gyro
Updated to version 5
This version greatly simplifies the code at the expense of a little processing inefficiency.
Instead of reading the file "PUPSTATE" directly from the savefile/savefolder, it looks for it in the frugal install directory. So it doesn't have to wait until the savefile/savefolder is available before making the final decision.
To use:
1) Put patched "initrd.gz" in the frugal install directory.
2) Reboot to show that everything is still the same.
3) Copy "/etc/rc.d/PUPSTATE" to the frugal install directory.
4) Add "nosearch=sda3" as boot parameter (using appropriate partition instead of "sda3").
5) Reboot.
The "nosearch" facility can be deactivated by either removing the "nosearch" boot parameter or deleting the "PUPSTATE" file in the frugal install directory.
To add an adrv or ydrv to the boot:
1) delete the copy of "PUPSTATE" in the frugal install directory.
2) Reboot.
3) Copy "/etc/rc.d/PUPSTATE" to the frugal install directory.
4) Reboot
See first post for download.
gyro
Instead of reading the file "PUPSTATE" directly from the savefile/savefolder, it looks for it in the frugal install directory. So it doesn't have to wait until the savefile/savefolder is available before making the final decision.
To use:
1) Put patched "initrd.gz" in the frugal install directory.
2) Reboot to show that everything is still the same.
3) Copy "/etc/rc.d/PUPSTATE" to the frugal install directory.
4) Add "nosearch=sda3" as boot parameter (using appropriate partition instead of "sda3").
5) Reboot.
The "nosearch" facility can be deactivated by either removing the "nosearch" boot parameter or deleting the "PUPSTATE" file in the frugal install directory.
To add an adrv or ydrv to the boot:
1) delete the copy of "PUPSTATE" in the frugal install directory.
2) Reboot.
3) Copy "/etc/rc.d/PUPSTATE" to the frugal install directory.
4) Reboot
See first post for download.
gyro
well that is the heart of the matter... fiddling with the window boxes is fine but sometimes its worth peeking in the basement at the foundations.Some of the complexity comes from trying to efficiently integrate it into the 'init' script.
Shinobar had the same problem...large scripts trying to cater for all the hoplessly messy puppyisms and variations.
My init in initrd is around 25k...thats a lot less crap to try and work around.
Its not a diff or 2 needed but a complete rewrite/re-evaluation.
I am sure gyro you could do a much better job yourself.
see whats needed...cut out things that are hopelessly obsolete and make a script to do the job...ps the cpio initrd adds to the complexity by the way. You could add nice features like better sfs handling.
mike