pUPnGO 2012

A home for all kinds of Puppy related projects
Message
Author
goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#201 Post by goingnuts »

No doubt busybox is needed. We could get through without it using ams-utils or the "multical call binary-technique" (mcb) of original apps...but BB delivers most of what is needed out of the box. Would be nice if they delivered some overview of which version is the most stable and content of apps between versions.
Also a "long term maintenance" version of BB could be nice. Upgrading BB (like upgrading other software) has the potential of breaking your existing scripts - so every script needs to be verified that it does what it is meant to do. On the other hand the new versions might offer apps that eliminate use of org apps and by then potentially reduce complexity and size.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#202 Post by technosaurus »

Ibidem wrote:
greengeek wrote:Does puppy HAVE to have busybox? What would happen to Puppy if it used a full command set?
Can you spell B-L-O-A-T?

To elaborate, I can build a ~700k static busybox that provides everything needed to boot (I have run this as an OS). This equates to:
coreutils:12 MB
bash: 800k
dhcpcd (127 k) or dhclient 643 k
net-tools: 927k
module-init-tools: 340k
netcat: 192 k
sysklogd: 200 k
iputils-ping: 131 k
netbase: 98 k
sed: 52 k
awk: 322 k
util-linux: 2 MB
e2fs-progs: 2 MB
wget: 2 MB
bzip2: 156 k
...
ie, at least 20 MB.
Also you get slower startup, more RAM used, etc.
I wrote a small c init that will boot into X with jwm that compiles to a few kb. It wouldn't be that much more to add more of the boot process in parallel if anyone is interested. Tinycore has a patch to allow the initrd to be swappable (the initramfs I was using isn't) as well as a couple of other kernel patches that will speed up booting. Basically all you need for the desktop is xvesa/xfbdev, jwm, rxvt and a shell, but I never made them into a threadsafe mcb, so they can just as well be separate static builds.

over 90% of the boot process that happens before x can be in parallel.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#203 Post by amigo »

"Does puppy HAVE to have busybox?" Yes, Puppy probably does. No one else should have to suffer a complete system loaded with cut-down versions of everything. Yeah, there's a big size difference, but I like using the same toys that the BIG boys use... LOL

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#204 Post by greengeek »

technosaurus wrote:I wrote a small c init that will boot into X with jwm that compiles to a few kb. It wouldn't be that much more to add more of the boot process in parallel if anyone is interested. Tinycore has a patch to allow the initrd to be swappable (the initramfs I was using isn't) as well as a couple of other kernel patches that will speed up booting. Basically all you need for the desktop is xvesa/xfbdev, jwm, rxvt and a shell, but I never made them into a threadsafe mcb, so they can just as well be separate static builds.
This sounds rather interesting. I fear it will take me a long time to get to grips with all the things I don't understand about the boot process and componentry, but the more I look at the simpler stuff, the more I am learning.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#205 Post by technosaurus »

to start X all that needs to happen is to mount /dev
to get a terminal emulator working, you also need to mount /dev/pts
and some (non-crucial) apps will need /sys and /proc mounted
this is one line of C each

While X is starting up you can load additional kernel modules, mount file systems, do checks, set settings etc... from kernel boot parameters (boot parameters should be the way most settings are stored so that you don't have to mount a file system to get them, but no-one else does it that way)

Once X and the wm are up, a terminal or gui setup wizard can be used for the rest (and jwm has a convenient startup tag... also restart and shutdown, but that is another topic)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#206 Post by greengeek »

technosaurus wrote:to start X all that needs to happen is to mount /dev
to get a terminal emulator working, you also need to mount /dev/pts
and some (non-crucial) apps will need /sys and /proc mounted
... you can load additional kernel modules, mount file systems, do checks, set settings etc... from kernel boot parameters
Once X and the wm are up, a terminal or gui setup wizard can be used for the rest (and jwm has a convenient startup tag...
I used the phrase "simpler stuff" but what I really meant was "more minimalistic stuff". Each time I read your posts I realise that the minimalistic stuff is way harder than the bigger stuff (like booting a fullblown puppy from CD...that I can do!) :-)

Understanding all this grassroots stuff about booting with the minimum required code is like getting to the moon on a bicycle. Enticing yet very difficult for the untrained :-)

Thank God for google!

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#207 Post by technosaurus »

The only reason it is complex is because we made it that way over time. For example puppy's init has tons of bloat dealing with finding the correct partitions, sfs and save file when it could just be added as a parameter to the kernel command line (same goes for kb, resolution, drivers to load, language and many others). Perhaps early bootloaders lacked this functionality and this is a holdover that exists because things still work? I don't know, but I literally spent days turning on and off kernel options to boil it down to the salts and then did the same with init. Once I realized how little was actually needed, it just made sense to write it in c since it was basically:

Mount /dev and /dev/pts
Fork and exec X
Set some basic env variables normally found in /etc/profile and DISPLAY
Fork and exec the wm
(Logic to reboot/shutdown/restartx)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#208 Post by Ibidem »

If you use Xorg or xfbdev, you'll need to load modules first.
And an autoconfigured X requires udev or hal. But then, most people use udev for /dev and loading modules.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#209 Post by technosaurus »

Ibidem wrote:If you use Xorg or xfbdev, you'll need to load modules first.
And an autoconfigured X requires udev or hal. But then, most people use udev for /dev and loading modules.
there _is_ a builltin option, IMHO everything needed by init _should_ be built-in unless there is some conflict (but in that case I would just have a separate version for the broken device)

Speaking of loading modules, there is a recent kernel patch to load modules by filename (previously they had to be memmapped) that would make my flat module method much easier in c (by flat I mean all modules in 1 dir). Even with standard tools it was faster this way because it didn't need to recurse into subdirs. I know this doesn't sound like much, but loading a single module previously required several lines of code, much of which was unnecessarily complex.

Speaking of udev, it has grown into a behemoth, and the things it does _can_ be simple. I'll probably try to patch toybox's mdev instead.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

PANZERKOPF
Posts: 282
Joined: Wed 16 Dec 2009, 21:38
Location: Earth

#210 Post by PANZERKOPF »

technosaurus wrote: Speaking of udev, it has grown into a behemoth, and the things it does _can_ be simple.
Agree.
technosaurus wrote: I'll probably try to patch toybox's mdev instead.
Busybox already has mdev, it can be used as simple udev replacement.
I made some scripts and mdev.conf file (Idea was borrowed from Alpine linux).
One problem was discovered there: It does not create device node for parallel port (LPT), althought all needed drivers was successfully loaded (parport, parport_pc).
Attachments
mdev.tar.gz
(1.59 KiB) Downloaded 307 times
SUUM CUIQUE.

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#211 Post by goingnuts »

PANZERKOPF: Never got mdev working...tried you scripts (deactivated udevd in init), run rc.mdev afterlogin. Had to add groups disk, uucp, cdrom and video. Then only minor warnings running mdev -s or rc.mdev.
But no drivers loaded afterwards...
When to launch rc.mdev? Do you need additional scripts/setup?

Btw. diethotplug is an other close to udevd replacement.

PANZERKOPF
Posts: 282
Joined: Wed 16 Dec 2009, 21:38
Location: Earth

#212 Post by PANZERKOPF »

goingnuts wrote:PANZERKOPF: Never got mdev working...tried you scripts (deactivated udevd in init), run rc.mdev afterlogin. Had to add groups disk, uucp, cdrom and video. Then only minor warnings running mdev -s or rc.mdev.
But no drivers loaded afterwards...
When to launch rc.mdev? Do you need additional scripts/setup?
Weird....
Usually mdev must be launched after mounting sysfs and proc.
Maybe Your busybox has different configuration?
Above code is a part of my system initscript. Attached tarball contains this script and
busybox configuration file.
Attachments
sysinit.tar.gz
(3.6 KiB) Downloaded 380 times
SUUM CUIQUE.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#213 Post by Ibidem »

Looking at the mdev scripts, I see a couple things...
1. mdev.conf is missing this section (from examples/mdev_fat.conf in the busybox source):

Code: Select all

# Syntax:
# [-]devicename_regex user:group mode [>|=path] [@|$|*cmd args...]
#
# =: move, >: move and create a symlink
# @|$|*: run $cmd on delete, @cmd on create, *cmd on both

# support module loading on hotplug
$MODALIAS=.*	root:root 660 @modprobe "$MODALIAS"
This is what handles hotplug module loading, which is important.

2. Most of the mknod stuff is unneeded with a proper mdev.conf.
I know /dev/{console,full,null,zero,urandom,random} are automatically created, even with mdev.conf absent.

3. sysctl -w kernel.hotplug=/sbin/mdev allows use wthout mounting /proc

4. Alpine uses/used something like:

Code: Select all

find /sys -name modalias |xargs sort -u |xargs modprobe -a -b
to load modules for devices that aren't hotplugged. This should really speed things up.
And to coldplug USB devices, you may need this:

Code: Select all

	# mdev -s will not create /dev/usb[1-9] devices with recent kernels
	# so we trigger hotplug events for usb for now
	for i in $(find /sys/devices -name 'usb[0-9]*'); do
		[ -e $i/uevent ] && echo add > $i/uevent
	done

Alpine's stuff is at:
http://git.alpinelinux.org/cgit/aports/ ... nitscripts

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#214 Post by greengeek »

PANZERKOPF wrote: Attached tarball contains this script and
busybox configuration file.
The rc.shutdown is very much simpler than some of the puppy shutdown scripts I have been looking at recently. Where would you use this? Is it taken from another distro or would it work in a cutdown puppy?

PANZERKOPF
Posts: 282
Joined: Wed 16 Dec 2009, 21:38
Location: Earth

#215 Post by PANZERKOPF »

Ibidem wrote:Looking at the mdev scripts, I see a couple things...
1. mdev.conf is missing this section (from examples/mdev_fat.conf in the busybox source):

Code: Select all

# Syntax:
# [-]devicename_regex user:group mode [>|=path] [@|$|*cmd args...]
#
# =: move, >: move and create a symlink
# @|$|*: run $cmd on delete, @cmd on create, *cmd on both

# support module loading on hotplug
$MODALIAS=.*	root:root 660 @modprobe "$MODALIAS"
This is what handles hotplug module loading, which is important.
We can use above trick as well as making script for particular event (see /lib/mdev/usbdev for example).
Ibidem wrote: 4. Alpine uses/used something like:

Code: Select all

find /sys -name modalias |xargs sort -u |xargs modprobe -a -b
to load modules for devices that aren't hotplugged. This should really speed things up.
And to coldplug USB devices, you may need this:

Code: Select all

	# mdev -s will not create /dev/usb[1-9] devices with recent kernels
	# so we trigger hotplug events for usb for now
	for i in $(find /sys/devices -name 'usb[0-9]*'); do
		[ -e $i/uevent ] && echo add > $i/uevent
	done

I learned Alpine scripts before making my one. That script unfortunately cannot detect
all devices plugged in my test boxes (like Broadcom wireless card).
Some devices hasnt modalias file in /sys directory but has uevent with modalias string inside.
Also, my version based on "pure shell", no grep, find, xargs etc. That is not important but
should be faster (IMHO) :)
SUUM CUIQUE.

PANZERKOPF
Posts: 282
Joined: Wed 16 Dec 2009, 21:38
Location: Earth

#216 Post by PANZERKOPF »

greengeek wrote:The rc.shutdown is very much simpler than some of the puppy shutdown scripts I have been looking at recently. Where would you use this? Is it taken from another distro or would it work in a cutdown puppy?
I use this in my own "pocket Linux", most things are written from scratch but some ideas
and functions were borrowed from other projects.
SUUM CUIQUE.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#217 Post by Ibidem »

PANZERKOPF wrote:
Ibidem wrote: 4. Alpine uses/used something like:

Code: Select all

find /sys -name modalias |xargs sort -u |xargs modprobe -a -b
to load modules for devices that aren't hotplugged. This should really speed things up.
And to coldplug USB devices, you may need this:

Code: Select all

	# mdev -s will not create /dev/usb[1-9] devices with recent kernels
	# so we trigger hotplug events for usb for now
	for i in $(find /sys/devices -name 'usb[0-9]*'); do
		[ -e $i/uevent ] && echo add > $i/uevent
	done

I learned Alpine scripts before making my one. That script unfortunately cannot detect
all devices plugged in my test boxes (like Broadcom wireless card).
Some devices hasnt modalias file in /sys directory but has uevent with modalias string inside.
Also, my version based on "pure shell", no grep, find, xargs etc. That is not important but
should be faster (IMHO) :)
A loop to load modules will probably be slower than loading all at once: you may save the time of invoking find/grep/xargs once, but you pay by reading each file serially and invoking modprobe once per module.
Plus, a for loop has some inherent overhead.

I timed the different methods with modprobe converted to an echo:
Anyway, here's an attempt to handle that case; there is some overhead to parsing the uevent files.
Brute force with find @ 0.39 seconds:

Code: Select all

find /sys/devices -name uevent | xargs grep -h MODALIAS 2>/dev/null |cut -d = -f 2 
Your way @0.14 seconds (goes down by half without the second /sys/bus/*):

Code: Select all

#!/bin/sh

#is there really a reason to try twice?
for ONEBUS in /sys/bus/* /sys/bus/*; do
    for ONEDEVICE in ${ONEBUS}/devices/*; do
	[ -r ${ONEDEVICE}/uevent ] || continue
	while read ONEUEVENTSTR; do
	    case $ONEUEVENTSTR in
		MODALIAS=*)
		    echo ${ONEUEVENTSTR#MODALIAS=} 2>/dev/null
		;;
		*)
		    continue
		;;
	    esac
	done < ${ONEDEVICE}/uevent
    done
done
And a mix @0.035 seconds:

Code: Select all

grep -h MODALIAS /sys/bus/*/devices/*/uevent 2>/dev/null |cut -d = -f 2
The "official" method is more like

Code: Select all

find /sys -name uevent -exec echo add > {} \;
but, that would likely have even worse performance (for every entry, a context switch to kernel mode and exec mdev in user mode, then exec modprobe if needed)

Each testcase outputs a list of module aliases.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#218 Post by Ibidem »

goingnuts wrote:PANZERKOPF: Never got mdev working...tried you scripts (deactivated udevd in init), run rc.mdev afterlogin. Had to add groups disk, uucp, cdrom and video. Then only minor warnings running mdev -s or rc.mdev.
But no drivers loaded afterwards...
When to launch rc.mdev? Do you need additional scripts/setup?

Btw. diethotplug is an other close to udevd replacement.
FYI: diethotplug is from 2002; you might want hotplug2.
Then there's mdev and ndev, and Busybox hotplug, and eudev, and probably a few other systems...
But udev/eudev provide one thing the other solutions don't: they can enumerate devices, which is what Xorg wants them for.

PANZERKOPF
Posts: 282
Joined: Wed 16 Dec 2009, 21:38
Location: Earth

#219 Post by PANZERKOPF »

Ibidem wrote: But udev/eudev provide one thing the other solutions don't: they can enumerate devices, which is what Xorg wants them for.
Yes, latest Xorg servers need udev but as I know, It can be configured/compiled without udev.
SUUM CUIQUE.

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#220 Post by goingnuts »

PANZERKOPF & Ibidem: Thanks for all your input! I guess I stick with udev-124 as its quite small and it works for me. Cant get BB mdev working, diethotplug is fast but lacks some features, hotplug2 seems to favor glibc-dynamic linking and eudev needs Autoconf version 2.68 or higher and I do not want to upgrade now...

And the udevd seems to work on kernel from P216, P412, P431 & P525 without recompile.

Revisited the static build of tcl/tk 8.5 to get attached 3 games running (bubbles.tcl CrystalsBattle.tcl gemgame.tcl) where the first & last will be known from Puppy3...quite addictive...
Attachments
3_tcl_games.tar.gz
3 classic tcl games
(27.39 KiB) Downloaded 303 times
Last edited by goingnuts on Mon 28 Jan 2013, 20:03, edited 1 time in total.

Post Reply