Busybox - fat and fast?

[MochiMoppel]

I was a bit shocked when radky wrote here that woof-CE has moved to busybox 1.30.1.

The lack of backward compatibility for the timeout command was one thing, but what I was most concerned of was the size. My slacko 5.6 comes with busybox v1.21.0. At 689K it is not really lightweight compared to single GNU commands, which typically weigh only around 50K. But now this new v1.30.1 is a real monster with its 958K.

I always assumed that running a busybox command must be slower than a slim GNU command and running a fat busybox version must be slower than running a leaner version.

Seems I was wrong. I tested the timeout command on all 3 versions:

For busybox 1.21.0

Code: Select all

time busybox timeout -t 1 yes

For busybox 1.30.1

Code: Select all

time busybox timeout 1 yes

For GNU timeout (coreutils 8.24)

Code: Select all

time timeout 1 yes

Results:

• busybox timeout (BusyBox v1.21.0; size 689K)
  real 0m1.006s
  user 0m0.037s
  sys 0m0.383s
  
  busybox timeout (BusyBox v1.30.1; size 958K)
  real 0m1.002s
  user 0m0.040s
  sys 0m0.490s
  
  timeout (GNU coreutils 8.24; size 47K)
  real 0m1.006s
  user 0m0.050s
  sys 0m0.480s

I also compared other commands. Not all showed dramatic differences, but I could see a tendency that newer busyboxes are faster than older (more efficient code?) and busyboxes are not slower than the coreutils. Surprising, but good to know

[Lobster]

Is busybox compressed with something like UPX?
https://upx.github.io/

[williams2]

Is busybox compressed with something like UPX?

Yes. Busybox is in the main Puppy sfs (squash file system). Everything in the squash file system is compressed.

There is not much point in having a upxed file in an sfs, it is already compresssed.

Older versions of upx did not permit multiple instances of an executable to share resources. I think the newer upx does permit sharing.

The busybox in bionicpup64 8.0 is compressed because it is in the main sfs file. It is about 938k, but it is larger because it is statically linked. That is, it does not need to use library files.

The first time busybox runs, it is copied and decompressed from the sfs file into a ram buffer. It is copied to cache space and copied into ram to be executed.

After that, any time busybox is run again, the system gets it already uncompressed from the cache memory in ram,

But busybox is already executing in ram. Init, which is busybox, and which starts the linux system is always in ram executing. And init runs several instances of getty. Getty is also busybox.

When two or more instances of a program are run, the system is smart enough to allow the identical parts of the program to be shared by each instance.

For example, you might have mpv playing a movie and you might have an urxvt window, and you might be looking at a text file in geany.

If you then open 2 or 3 more urxvt windows, and open 2 or 3 more geany windows, and run another movie in mpv, the system does not need to copy urxvt or geany or mpv to ram again. It can use the first copies that are still in ram.

Because busybox is always executing in ram, running a busybox utility program is very efficient. The executable is already in ram, ready to be used. So it is fast and takes up very little space.

For example, opening a urxvt window and typing ash shows ash using 4k of ram, according to htop. And getty is using 4k each.

Another thing, because busybox was compiled statically, and does not need any library files, it should run in almost any linux, 32 bit or 64 bit. And linking to library files is slower than not needing to link.

[Lobster]

Thanks @Williams2 very comprehensive explanation, my next question was going to be about multiple instances ...

[musher0]

Except that, compared to the originals, some utilities in busybox are so trimmed down,
they are almost useless: take the busybox less and the busybox lzop, for example.

There may be a new busybox out, but its principle remains the same: chop-chop-chop the
original utility until it's a ghost ot its former self.

If you sing the praise of busybox, also hurry up and install a recent copy of the GNU
coreutils & co, in case some busybox utility can't do the job the original does.

IMO, busybox has a decent implementation of ash -- if you want speedy execution for an
uncomplicated script. That's about it.

[rufwoof]

:%d

[musher0]

Hi rufwoof.

busybox is useful to cross the "initrd border". After that, its usefulness depends on the
user's needs. It may be sufficient for the needs of a power user such as yourself, but
run-of-the-mill users need to know the complete story about busybox.

IMO, it's NOT

functional enough for general usage,

as you mention. Unless one lets the people at busybox to decide what "general usage"
is for the rest of us.

The example I always give to prove this point is: the busybox less has four lines of on-
line help, whereas the real less by greenwood software has four pages of on-line help.

Where did 95% of the functions go in the busybox less? In the shredder, because
someone at busybox thought, for example, that less' capacity to read many texts at
once and offer the user the capacity to go from one text to the other, wasn't worth it --
and so on for the other functions of less (show ANSI colors, highlight text, etc.).

While the busybox less may be a programming feat, it is giving the wrong idea about
the real less, and spreading fake news about what the real less can do for the user.

And so on for lzop, mount, etc., and all the utilities busybox pretends to be replacing.

I think the conclusion is obvious: the user should be made aware of what busybox is:
it's useful in the limited initrd context, and that's it.

To users: if you feel it would be nice that a busybox utility had a certain feature,
please go look for the original utility: it probably has that that feature and busybox
is probably short-changing you!

BFN.

[MochiMoppel]

When two or more instances of a program are run, the system is smart enough to allow the identical parts of the program to be shared by each instance.

I understand this, but what I don't understand:
My system starts with the old 689K bb and keeps it running. When I then - as in my test - start a new 958K bb, I would expect that this is not treated as merely another instance of the already running version. After all the new bb contains considerable changes and it is bigger and therefore additional stuff has to be loaded. I don't see how starting a new and different version can result in faster execution when compared to starting exactly the same version that is already running.

[BarryK]

I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.

It has been changed to work more like the full dc, which will break some scripts that use the busybox version.

But the real problem is that it does not always work. It is supposed to do arithmetic on numbers with fractions, but I found that in certain cases it truncated the fractional part of the result,

I can't recall the details.

I did upgrade busybox for the initrd, but do not use dc in that situation.

[MochiMoppel]

I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.

Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.

[rufwoof]

:%d

[MochiMoppel]

@rufwoof: How does this relate to the thread topic?

[technosaurus]

In case nobody else noticed, it is now possible to embed shell scripts in busybox.

Advantages
For the initrd/initramfs, that saves close to 4kb and 1 inode per small script
It is still compressible (on squashfs or using upx)
It should also give faster access times to scripts especially for full installs.
/bin/sh can be linked to bash and busybox compatible scripts can still run faster
Only 1 file to distribute

The downside:
Larger .text section in busybox (though smaller in total)
Scripts would no longer be user editable
Requires busybox rebuild to fix a script

Just for reference, a script I use to pull out all libs used by a program - so you can drop them into initrd ...

What is this tinycore stuff? We do static builds for initrd binaries... could be useful for containers though (as in Barry's EasyOS)

I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.

Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.

select N for the "big" version of dc if you want the old one that is not based on the new bc code (IMHO the bc code needs about 5-10 more cleanup passes before it should be a default applet anyhow)

[BarryK]

I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.

Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.

Ah, that's good. Thanks for the info.

[BarryK]

I haven't upgraded busybox to 1.30.x in EasyOS, as the 'dc' applet is broken.

Was broken in 1.30.0
1.30.1 is a bugfix release that contains "fixes for bc/dc". May solve your problems.

select N for the "big" version of dc if you want the old one that is not based on the new bc code (IMHO the bc code needs about 5-10 more cleanup passes before it should be a default applet anyhow)

That's good too, if I can keep the old behaviour of dc.

Interesting about the embedded scripts.

[sc0ttman]

Interesting about the embedded scripts.

Agreed - if the scripts embedded inside busybox also get a performance boost by being there...

Pkg is an ash script, so it should work nicely with ash inside busybox.. probably... after a few fixes

[s243a]

:%d

I should have been following this thread closer. It looks like I missed the discussion