patches to time boot messages to help to speed up boot

[manelapb]

Hi everyone,

I attach some patches for Puppy 5.20 to time boot messages (in screen and to log) and to have a single boot log with uptimed messages.
Is this the right place for this?

The idea is to locate wasted time to help to speed up boot.

I boot from usbstick with "pmedia=usbflash pfix=ram,nocopy pkeys=es"
Update 13.3.11: I add attachment with v2

here a grep "^\[" /tmp/bootsysinit.log

Code: Select all

[ 1.37 ] Loading 'es' keyboard layout...
[ 1.38 ] Loading drivers needed to access disk drives
[ 4.55 ] - Waiting for USB device to register
[ 11.56 ] - Waiting for usb partitions
[ 28.59 ] Searching for Puppy files in computer disk drives...
[ 28.77 ] Loading the 'lupu_520.sfs' main file...
[ 28.90 ] Setting up the layered filesystem...
[ 28.90 ] Overlaying preconfig files...
[ 29.06 ] Performing a 'switch_root' to the layered filesystem:
[ 29.07 ] - Copying files and sync...
[ 30.52 ] - switch_root... (End of init in initrd.gz)
[ 31.00 ] Making the filesystem usable...
[ 34.56 ] Updating...
[ 41.51 ] Loading kernel modules...
[ 46.50 ] Loading swap partition /dev/sda3...
[ 46.59 ] Waiting for modules to complete loading...
[ 48.64 ] Setting up services (network, printing, etc.)...
[ 48.69 ] Recognising media devices...
[ 49.82 ] End of rc.sysinit, next /etc/profile (after root login)
[ 50.15 ] Starting xwin script to run X windows
[ 50.77 ] Starting X, specs in /etc/X11/xorg.conf, startup apps /root/.xinitrc...
[ 53.29 ] X server started, running /root/.xinitrc
[ 54.38 ] Starting Window Manager
[ 56.64 ] X desktop started, running /root/Startup scripts

You can convert that data to csv for spreadsheet process with:

Code: Select all

#!/bin/sh
#    Use: bootmsg2csv bootsysinit-copy.log, or just bootmsg2csv  
# parse log, get bootmsg messages and convert to CSV for spreadsheet processing

FILE=$1
[ ! "$FILE" ] && FILE="/tmp/bootsysinit.log"
grep "^\[" $FILE | tr "[]," " , " >$FILE.csv
grep "^\[" $FILE	# Also show

Here the more wasting steps:
17.03 - Waiting for usb partitions
7.01 - Waiting for USB device to register
6.95 Updating...
4.99 Loading kernel modules...
3.56 Making the filesystem usable...
3.17 Loading drivers needed to access disk drives

I think first one is a bug (I will report it), and at least Updating... can be avoid or speed up greatly...

[technosaurus]

You may try bootchartd from recent busybox.

[manelapb]

You may try bootchartd from recent busybox.

Thanks for the tip, and I would like to have some form of my patches in the official Puppy to help to take boot time into account in the development and use, so I prefer to stay with what is present inside bare Puppy 5.20

I would like to have people to pay some attention to the boot time, I expect I success...

[Karl Godt]

nothing against your way of getting information, but it might be that the puppy kernel supports the
printk.time=1 kernel bootparameter, which would look like

bash-3.00# tail /tmp/bootkernel.log
<6>[ 58.623065] it87: Found IT8712F chip at 0x290, revision 4
<6>[ 58.650535] it87 it87.656: Detected broken BIOS defaults, disabling PWM interface
<6>[ 59.119446] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
<6>[ 61.980595] ip_tables: (C) 2000-2006 Netfilter Core Team
<4>[ 62.083572] nf_conntrack version 0.5.0 (7167 buckets, 28668 max)
<4>[ 62.116306] CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
<4>[ 62.134230] nf_conntrack.acct=1 kernel paramater, acct=1 nf_conntrack module option or
<4>[ 62.152106] sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
<6>[ 64.317083] lp0: using parport0 (interrupt-driven).
<6>[ 64.334758] lp0: console ready
bash-3.00#

btw : I disabled the pretty output for busybox dmesg ; kernel is a 2.6.30 series and nothing to worry bout ip_tables

[manelapb]

nothing against your way of getting information, but it might be that the puppy kernel supports the
printk.time=1 kernel bootparameter...

Thanks, that is a good tip for modules boot timing research. I have tested it and it only prints times in bootkernel.log nothing in the logs of init and rc.sysinit

[nooby]

I know nothing about these things but I do know that if puppy did not wait for the USB then the hardware would not have time to settle an send correct signals and it would fail.

So sure you could somehow have a boot POST thing in BIOS that test if some USB are attached or not and that the boot up thing knows that and look there for such and skip the waiting if there are no USB attached.

I am not good at explaining but the waiting are there for a reason.
sure it is a compromize because how does one know it there is a usb or not.

Another way would be to tell the user of a OS to write explicit code on the kernel line. "Don't look for USB I have none attached" then that would speed up boot. And many other such things could be specified at boot time and if set up once then the OS could boot superfast because it is tailored to the machine and not a general booting OS that need to find out what machine it is on.

But you do know such things better than me.

I found a guy that had take ExpressGate from the company and tried to make a program out of it that boot real fast. But I was too lazy to try to grasp all the tech details and besides I don't think the owners are happy about such prying usage of their commercial efforts. But it show to what length people can go to boot fast. To backingeneer what others have as trade secret.

[technosaurus]

With some hacking of pupngo I was able to get to the xvesa + jwm desktop in ~3 sec on a 1.6ghz machine. It wasn't by any means fully functional, but the rest could load in the background ... possibly even in parts.

[manelapb]

... if puppy did not wait for the USB then the hardware would not have time to settle an send correct signals and it would fail.

take a look to http://www.murga-linux.com/puppy/viewto ... 607#504607

Another way would be to tell the user of a OS to write explicit code on the kernel line. "Don't look for USB I have none attached" then that would speed up boot.

Hummmmm...

[manelapb]

With some hacking of pupngo I was able to get to the xvesa + jwm desktop in ~3 sec on a 1.6ghz machine. It wasn't by any means fully functional, but the rest could load in the background ... possibly even in parts.

After taking a look at it now I strongly believe that a 15 seconds USB boot of a nearly standard Puppy (with some tuning boot parameters) is really possible, with just small modifications to the boot scripts.

[Dougal]

... if puppy did not wait for the USB then the hardware would not have time to settle an send correct signals and it would fail.

take a look to http://www.murga-linux.com/puppy/viewto ... 607#504607

Another way would be to tell the user of a OS to write explicit code on the kernel line. "Don't look for USB I have none attached" then that would speed up boot.

Hummmmm...

Code: Select all

echo 0 > /sys/module/usb_storage/parameters/delay_use

tends to make things work faster. 0 can cause problems with devices not being detected, in which case 1 should be ok.
(I added to my init script a usb_delay= parameter to control how many seconds to use).

[jemimah]

With some hacking of pupngo I was able to get to the xvesa + jwm desktop in ~3 sec on a 1.6ghz machine. It wasn't by any means fully functional, but the rest could load in the background ... possibly even in parts.

You booted a normal kernel in less than 3 seconds? Are there details posted somewhere?

[technosaurus]

[quote="jemimah"]
You booted a normal kernel in less than 3 seconds? Are there details posted somewhere?
[/quote]
Yes, normal kernel, but the first (almost) line in init was a variation of
Xvesa -screen 640x480x16 -nolisten tcp & sleep .1 && jwm -display :0 &
(Using a static uclibc xvesa and jwm in the initrd.)
I could not get a terminal emulator in that environment to work though... not sure if I needed to have getty running or mount something or export variables???iirc it said something along the line of ...can't open pty ...but looking back this may be due to my uclibc config... Rob has some pty related options disabled in the default aboriginal config.

[b]edit[/b] the plan was to give the load status by using a tray(s) to update load status by doing something like

[code]
ln -sf $HOME/.jwm/loading.modules $HOME/.jwmrc-tray
jwm -restart
...
ln -sf $HOME/.jwm/mounting $HOME/.jwmrc-tray
jwm -restart
...
[/code]

[technosaurus]

I'm not sure why my last post isn't formatting...

Anyhow, once you have the times for each process, you would still need to do a few things.

Analyze each process to determine it's predecessors and (processes that NEED to happen before it)
Determine the critical path (longest time from start to finish via predecessors)
Arrange the remaining processes with the longest ones starting first.(spawn them with "&" if they don't have a successor)

There still may be bottlenecks due to maxed out CPU (this is where tools like bootchartd come in ... to help with resource leveling) for instance - some parts of your critical path may not use much CPU, while other sections are CPU intensive. Ideally you would want to run the high CPU using non-critical items during the least CPU intensive part of your critical path and vice versa.... basic project management stuff really. I've been waiting for a decent gtk project management application to magically make itself known to me... any ideas... amigo mentioned one a while back, but I don't recall the name.

[jemimah]

I searched high and low for a project management app a few months ago with no good result. Most of the recommendations were for cloud-based apps.

---

You can't really do much of anything until the kernel modules have completed loading. Using a custom mostly-monolithic kernel with a full install and using the zdrive cutter to remove unneeded modules is the easiest (haha) way to get significant gains on boot speed. Removing unneeded code from the sysinit script helps too.

[technosaurus]

I searched high and low for a project management app a few months ago with no good result. Most of the recommendations were for cloud-based apps.

The closest I could find was Planner ( http://live.gnome.org/Planner ), which does not appear to be very active and has some since deprecated Gnome dependencies (gtk needs a good database and presentation tool too.)
However the dev version no longer requires gnome-vfs, libbonoboui, libgnomeprint, or libgnomeprintui ... only libgnomecanvas, libgnomeui, libglade-2.0 & gconf

You can't really do much of anything until the kernel modules have completed loading. Using a custom mostly-monolithic kernel with a full install and using the zdrive cutter to remove unneeded modules is the easiest (haha) way to get significant gains on boot speed. Removing unneeded code from the sysinit script helps too.

The one thing you _can_ do is run Xvesa and jwm (vesa support is built in by default), I probably need to step through the boot process somehow to figure out what part(s) make(s) the terminal emulator usable b/c jwm will not restart unless the command is issued from within the session however "rxvt -e ..." will automagically attach itself to work around that limitation

[manelapb]

(I added to my init script a usb_delay= parameter to control how many seconds to use).

Yes, this is the way...

usb_delay=0 failed to me until I hacked an ugly waitdev=sdb1 boot parameter to wait directly the drive I want.

That and after fixing a 2 seconds delay because there is no floppy in my PC and I arrive to "Performing a 'switch_root'" in 3.00 seconds

From 29.06 to 3.00 seconds is a quite great improvement!

[jemimah]

The one thing you _can_ do is run Xvesa and jwm (vesa support is built in by default), I probably need to step through the boot process somehow to figure out what part(s) make(s) the terminal emulator usable b/c jwm will not restart unless the command is issued from within the session however "rxvt -e ..." will automagically attach itself to work around that limitation

I wonder if that's how the "splashtop" OSes are doing it - just basic VESA graphics and not much else.

[technosaurus]

I wonder if that's how the "splashtop" OSes are doing it - just basic VESA graphics and not much else.

thats how most of the "big boys" do it - they pretend to have a functional desktop while in fact things are still loading in the background.... but no, actually you can run a lot of apps from the framebuffer (gtk2 has a directfb port and a linuxfb port up until 2.14) most everything is in the initial ramdisk so they don't need to wait on drive module loading and mounting to get OS files (Which is what I did on my attempt) You could probably get a static uclibc Midori up in ~1 sec... Minimizing the size helps too (for load/decompression time) Which reminds me, I have been meaning to ask you what you thought would be the minimal set tools of tools for internet connection (I am already porting iwlist and iwconfig to busybox) maybe wpa_supplicant_cli? (I think iwconfig in wireless tools dev may handle it ok now though?)

[jemimah]

AFAIK wpa_supplicant is still needed to make wpa connections.

Frisbee also needs dhcpcd - it could probably be fixed to deal with busybox dhcp client though IMO, dhcpcd at 75k is worth it.