busybox extras

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#61 Post by Karl Godt »

First issues :

1) grep -w option :

~ # df | grep -w '/'
~ # df | /bin/grep -w '/'
/dev/root 15116868 12621452 1727512 88% /


2) sed problems with -i option

bash-3.2# echo HALLO >testfile.hallo
bash-3.2# busybox sed "1 a\ " testfile.hallo
HALLO

bash-3.2# busybox sed -i "1 a\ " testfile.hallo
sed: -i requires an argument
OR
sed: unsupported command u
for
sed -i "$Line a\ " "$FILE".sorted

bash-3.2# busybox_1.18.3_STATIC_upx9_648KB sed -i "1 a\ " testfile.hallo
works OK .
«Give me GUI or Death» -- I give you [[Xx]term[inal]] [[Cc]on[s][ole]] .
Macpup user since 2010 on full installations.
People who want problems with Puppy boot frugal :P

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#62 Post by musher0 »

jamesbond wrote:
musher0 wrote:You can't pass a Coke for a Pepsi, and vice versa. I wish someone on this
forum would have told me the true nature of busybox: it would have
saved me days of research on the Internet for the more complete
utilities i needed -- as well as a feeling of having been duped.
From busybox official homepage http://busybox.net/about.html
About Busybox wrote:BusyBox combines tiny versions of many common UNIX utilities into a single small executable. It provides replacements for most of the utilities you usually find in GNU fileutils, shellutils, etc. The utilities in BusyBox generally have fewer options than their full-featured GNU cousins; however, the options that are included provide the expected functionality and behave very much like their GNU counterparts. BusyBox provides a fairly complete environment for any small or embedded system.

BusyBox has been written with size-optimization and limited resources in mind. It is also extremely modular so you can easily include or exclude commands (or features) at compile time. This makes it easy to customize your embedded systems. To create a working system, just add some device nodes in /dev, a few configuration files in /etc, and a Linux kernel.
(emphasis is mine). And it has been like that from day one. The reason why Puppy uses busybox is because of that small size. Puppy only uses the full GNU utilities when it is absolutely necessary. That's one of the reason why Puppy stays small when other distro grows by leaps and bounds.

(...)
@jamesbond.

Thanks for pointing it out. I believe that "warning" should be advertised in an obvious place in Puppy as well, even if the Puppy newbie may not know the full extent of its meaning.

Regards.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#63 Post by Ibidem »

Karl Godt wrote:First issues :

1) grep -w option :

~ # df | grep -w '/'
~ # df | /bin/grep -w '/'
/dev/root 15116868 12621452 1727512 88% /


2) sed problems with -i option

bash-3.2# echo HALLO >testfile.hallo
bash-3.2# busybox sed "1 a\ " testfile.hallo
HALLO

bash-3.2# busybox sed -i "1 a\ " testfile.hallo
sed: -i requires an argument
OR
sed: unsupported command u
for
sed -i "$Line a\ " "$FILE".sorted

bash-3.2# busybox_1.18.3_STATIC_upx9_648KB sed -i "1 a\ " testfile.hallo
works OK .
grep appears to be a busybox issue wherein / is a word delimiter; sed -i is a musl one (workaround: sed -i "" ...) which I'm not sure about just yet.

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#64 Post by Karl Godt »

Yep , grep -w '/' is busybox regular source specific and sed -i compiled with regular libc works with 1.22.0.git and backwards tested until 1.18.5 .

Another thingys with the regular busybox :

1)readlink: illegal option: e

bash-3.00# busybox readlink has these options
BusyBox v1.22.0.git (2013-06-22 02:42:56 GMT+1) multi-call binary.
Usage: readlink [-fnv] FILE
Display the value of a symlink
-f Canonicalize by following all symlinks
-n Don't add newline
-v Verbose

while

bash-3.00# readlink --help
Usage: readlink [OPTION]... FILE
Display value of a symbolic link on standard output.
-f, --canonicalize canonicalize by following every symlink in
every component of the given name recursively;
all but the last component must exist

-e, --canonicalize-existing canonicalize by following every symlink in
every component of the given name recursively,
all components must exist
-m, --canonicalize-missing canonicalize by following every symlink in
every component of the given name recursively,
without requirements on components existence
-n, --no-newline do not output the trailing newline
-q, --quiet,
-s, --silent suppress most error messages
-v, --verbose report error messages

has those options .

2)
Also regular busybox ls is missing the --hide option as it is used in the (probably old now) ash script /sbin/pup_event_frontend_d :
(with workaround)

Code: Select all

#ls -1 --hide=ram* --hide=loop* /sys/block > /tmp/pup_event_frontend_block2
 ls -1 /sys/block |grep -vE 'ram*|loop*|md*|mtd*|nbd*' > /tmp/pup_event_frontend_block2
3)
modprobe could need support for the --config option, as it is used in rc.sysinit
-C <config-file> --config

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#65 Post by jamesbond »

Ibidem wrote:Looking through your config, I see a few things I'm curious about...
Try setting CONFIG_FEATURE_COMPRESS_USAGE; it compresses the usage messages, which can be a significant gain.
Yes, I should have enabled this.
I think you could disable CONFIG_FEATURE_HAVE_RPC, since only inetd cares; but it doesn't change much of anything.
I can't remember what this is for, but my busybox has inetd applet, so isn't it needed?
While rpm2cpio is useful, you might want to disable rpm.
I'll keep rpm for the fun of it :) (though apparently there are a few versions of RPM and the new ones can't be opened with busybox rpm). I have dpkg-deb included too :D
Why set CONFIG_FEATURE_OSF_LABEL?
You're right, I should disable this.
I don't know about others, but I use XFS.
I don't, but I like to support it for others who do. What's the config that I missed?
I doubt many Puppy users need CONFIG_MT; it's for controlling magnetic tape drives.
:lol: You're probably right :lol:
While I haven't done so yet, you may wish to disable CONFIG_HUSH; it's a second shell, and it's more minimal than ash.
Yes, now that I think about it, since I already have "ash" included, there is no point having "hush". In any case these days I go with standalone "dash" for most of my scripts. 74k statically compiled with klibc (or is it with musl? I can't recall), loads in no time, runs like the wind.

Thanks for running through the config and the constructive comments. I'm waiting for 1.22 to then I'll rebuild with all these suggestions (plus runit tools too).
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#66 Post by Ibidem »

jamesbond wrote:
Ibidem wrote:Looking through your config, I see a few things I'm curious about...
Try setting CONFIG_FEATURE_COMPRESS_USAGE; it compresses the usage messages, which can be a significant gain.
Yes, I should have enabled this.
I think you could disable CONFIG_FEATURE_HAVE_RPC, since only inetd cares; but it doesn't change much of anything.
I can't remember what this is for, but my busybox has inetd applet, so isn't it needed?
You disabled RPC support in inetd, so no. It tells the system that you have the RPC headers available and want to use them.
While rpm2cpio is useful, you might want to disable rpm.
I'll keep rpm for the fun of it :) (though apparently there are a few versions of RPM and the new ones can't be opened with busybox rpm). I have dpkg-deb included too :D
rpm2cpio is what I'd expect a Puppy to use, since the dependencies are not likely to be helpful. The rpm applet is an actual (hackish) package manager. That's all.
Why set CONFIG_FEATURE_OSF_LABEL?
You're right, I should disable this.
I don't know about others, but I use XFS.
I don't, but I like to support it for others who do. What's the config that I missed?
CONFIG_FEATURE_VOLUMEID_XFS
I doubt many Puppy users need CONFIG_MT; it's for controlling magnetic tape drives.
:lol: You're probably right :lol:
While I haven't done so yet, you may wish to disable CONFIG_HUSH; it's a second shell, and it's more minimal than ash.
Yes, now that I think about it, since I already have "ash" included, there is no point having "hush". In any case these days I go with standalone "dash" for most of my scripts. 74k statically compiled with klibc (or is it with musl? I can't recall), loads in no time, runs like the wind.

Thanks for running through the config and the constructive comments. I'm waiting for 1.22 to then I'll rebuild with all these suggestions (plus runit tools too).
If dash works, try busybox ash wherever the busybox applets would do the job; it will speed things up even more, due to not loading external programs.

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#67 Post by Karl Godt »

CONFIG_FEATURE_VOLUMEID_XFS
Unfortunately xfs.ko might be compiled as module in Standard Puppies (until now) and is not packed into the initrd.gz ( same with jfs.ko ) .
ibidem, if you want to make a Puppy derivate, maybe a good starting point would be a Xfs-Puppy , probably by a ^^simply^ remaster^ .

But it is also quite easy to miss some configuration options in the huge make menuconfigs of programs or kernel . :lol:

*

Another Wish-List (also for standard coreutils) :

tty : Option for to show only major,minor numbers like in ps -o tty .

ps : -o format option (like tname) to show tty-name, not tty-major,minor .
ps does not complain on some regular ps -options (ignores them) eg -A or -f , but chokes on -H option .

ls : Implemention of -o format option to -l option , eg I would like ls -l output like this :

Code: Select all

tty1 4,1 c rw-,---,--- root,root Jun,24,09:54
to have it easier to pipe it through awk .
BTW to praise busybox also : I like the ls coloration for /dev nodes (pink), better than yellow on white bg .

*

Personal update :

Ash scripts
rc.sysinit
pup_event_frontend_d
pup_event_backend_modprobe
rc.shutdown (without having tested the savefile creation)
are (fixed) working with ash preferring applets ( sometimes by calling /bin/path/regular.bin ) .

Ported to ash :
xwin
probepart
probedisk2
with no or nearly none adjustments.

So the boot to desktop works and also shutdown . Quite nice :D

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#68 Post by Karl Godt »

Short update on musl-busybox : getopt does not work for that line in /usr/sbin/alsaconf :

Code: Select all

OPTS=`busybox-orig getopt -o lmL:hp:Pu:g:d:rs:c:R --long legacy,modinfo,log:,help,probe:,listprobe,uid:,gid:,devmode:,strict,sound:,config:,resources -n alsaconf -- "$@"` || exit 1
workaround is to use ^busybox-orig^ that came with Puppy by default .
«Give me GUI or Death» -- I give you [[Xx]term[inal]] [[Cc]on[s][ole]] .
Macpup user since 2010 on full installations.
People who want problems with Puppy boot frugal :P

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#69 Post by Karl Godt »

Next :

1) modprobe -v does not work as it does in regular /sbin/modprobe to show inserted modules . Bad for debugging purposes .

2) modinfo :
# modinfo dummymodule;echo $?
filename: dummymodule
0
# modprobe -l | grep dummymodule
:lol:

3) chat : Takes much longer time to finish than regular chat . It also does interpret "\n" as literal opposite to regular chat, which shows a newline .

4) find : Puppy uses -noleaf option once a while . Many times in combination with -mount . -mount seems to be ignored or used as -xdev , not sure about it . At least bb find does not complain about -mount .

5) ash : /etc/rc.d/rc.firewall is using arrays .

*

What is making scripter's live uneasy in GNU/Linux :

1) rdev :
Why the hell append "/" ?

2) mount and df using /dev/root :
Who needs "/dev/root" ?
I cannot remember the kernel compiled with devtmpfs would create it .
Puppy's df script is a wrapper to change /dev/root to /dev/sdaX using rdev . It's used a lot in scripts in full installation .

To create /dev/root manually one could use rdev, parse /proc/cmdline for it or use dmesg output like

Code: Select all

dmesg | grep 'Mounted root'| grep -o 'device.*' | tr -d '.$' | cut -f 2 -d ' '
and then doing some brilliant long shell gymnastics to transform major:minor into /dev/sdaX .

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#70 Post by Ibidem »

Karl Godt wrote:Next :

1) modprobe -v does not work as it does in regular /sbin/modprobe to show inserted modules . Bad for debugging purposes .

2) modinfo :
# modinfo dummymodule;echo $?
filename: dummymodule
0
# modprobe -l | grep dummymodule
:lol:
I noticed that while working on toybox modinfo, and then I tried reading busybox modinfo so I could patch it...
It may not be really bad, but http://git.busybox.net/busybox/tree/modutils/modinfo.c looks bad after http://landley.net/hg/toybox/file/859a9 ... /modinfo.c

Looking at it again, I think lines 70-72 need either a test on line 70 or relocation after line 85.

I've also noticed they don't support -k (semi-easy) or -b (potentially harder); there are a lot of places that would need a change for these to work.
3) chat : Takes much longer time to finish than regular chat . It also does interpret "\n" as literal opposite to regular chat, which shows a newline .

4) find : Puppy uses -noleaf option once a while . Many times in combination with -mount . -mount seems to be ignored or used as -xdev , not sure about it . At least bb find does not complain about -mount .
Of course GNU would make an internal optimization into an option...
-noleaf is only relevant for GNU find; busybox find should ignore it because they don't use the same trick.

5) ash : /etc/rc.d/rc.firewall is using arrays .

*

What is making scripter's live uneasy in GNU/Linux :

1) rdev :
Why the hell append "/" ?

2) mount and df using /dev/root :
Who needs "/dev/root" ?
I cannot remember the kernel compiled with devtmpfs would create it .
Puppy's df script is a wrapper to change /dev/root to /dev/sdaX using rdev . It's used a lot in scripts in full installation .

To create /dev/root manually one could use rdev, parse /proc/cmdline for it or use dmesg output like

Code: Select all

dmesg | grep 'Mounted root'| grep -o 'device.*' | tr -d '.$' | cut -f 2 -d ' '
and then doing some brilliant long shell gymnastics to transform major:minor into /dev/sdaX .
1) Because it prints "an /etc/mtab line".

2) That's something that Rob Landley was working on a fix for.
Then, when Busybox relicensed to GPL2 only because there was quite a bit of code that really was only GPL2, Bruce Perens popped up, complained that GPL2 "or later" did not mean that you could relicense to GPL2 only, and also asked what GPL2 only gave them (having missed the discussion).
So the new "df" ended up as the first applet in toybox.

Note that porting from toybox to busybox is not exactly straightforward; toybox provides a god bit more infrastructure support. For example, toybox parses the arguments automatically (USE_<APPLET>(...)), doesn't need anything for kconfig (automatically picks up the right files), gets menuconfig help and usage from the same place, has the dirtree_*() functions, GLOBALS() for storing arguments passed to each parameter, etc...
See http://landley.net/hg/toybox/file/tip/toys/posix/df.c if you're curious about it.

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#71 Post by Karl Godt »

myself wrote:Yep , grep -w '/' is busybox regular source specific and sed -i compiled with regular libc works with 1.22.0.git and backwards tested until 1.18.5 .
Ha, found a way to use busybox grep -w on :punct: :

ash
BusyBox v1.22.0.git (2013-06-22 02:42:56 GMT+1) multi-call binary.
-F PATTERN is a literal (not regexp)

Code: Select all

[puppypc]# grep -Fw '/' /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext4 rw,relatime,barrier=1,data=ordered 0 0
«Give me GUI or Death» -- I give you [[Xx]term[inal]] [[Cc]on[s][ole]] .
Macpup user since 2010 on full installations.
People who want problems with Puppy boot frugal :P

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

busybox find is 2-3 times slower than regular find

#72 Post by Karl Godt »

Just to have a bookmark in the cloud :
recursive_action (and thus find) slow due to [l]stat()
Rich Felker Tue, 28 May 2013 21:13:18 -0700
Conceptually, the find utility need not perform lstat on each filename
unless it's needed for matching criteria. However, find is implemented
based on libbb's recursive_action, which always performs stat or
lstat. This makes busybox's find excruciatingly slow compared to GNU
find.
https://www.mail-archive.com/busybox@bu ... 18342.html
:shock:

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#73 Post by Karl Godt »

Hmm ...
http://www.busybox.net wrote: 20 January 2014 -- BusyBox 1.22.1 (stable)
grep (fixes for -w handling)
seems to work,
but not when compiled with musl :

Code: Select all

USING
/bin/busybox-1.22.1-dyn-musl
bash-3.00# echo "$S" | busybox grep -w B
B
BB
B

bbconfig | grep musl
CONFIG_CROSS_COMPILER_PREFIX="/usr/musl/bin/musl-"
CONFIG_SYSROOT="/usr/musl/include"
Also here grep grabs StringString...
Should only grep word B - not word BB as compiled with regular glibc does .

musl is still 9.11 from June 29, 2013 - tell me if it is worth to upgrade
http://www.musl-libc.org/download.html

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#74 Post by Ibidem »

A newer musl has several bugfixes; unfortunately, they won't fix that problem. (I believe they require the GNU regex API for the "fixed" grep -w.)
I suppose one could add some trickery so as to check if string[pmatch[n]->rm.so - 1] is a word boundary and string[pmatch[n]->rm.eo] is also a word boundary...

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#75 Post by Karl Godt »

Short note :
Compiled with musl 0.9.11 ,
the 'top' applet of BB segfaulted , when CONFIG_STATIC=y
- worked ok as dynamic
- but dyn is 4KB bigger than with regular GLIBC .

Have upgraded to musl 1.0.1 and 'top' is OK now .

Post Reply