180 small static dietlibc programs

[technosaurus]

I compiled almost the entire trunk of fefe.de, the home of dietlibc.

The results are in an tgz with a usage folder that tells how to use a lot of the commands. Many did not have a make install so I am uploading all of the stripped static binaries. See fefe.de or your favorite search engine for further usage

The best find is a little web server called gatling. Most of the rest are similar to busybox commands (only individual vs symlinks).

I am slowly building a whole set of dietlibc static binaries on my way to seeing if I can build a recent dietX

total 3516k
4312 acc
14976 acl
8476 allinone
1088 arch
3784 arping
2144 auto-str
4724 avcache
4724 avcache-postfix
42432 badblocks
1216 basename
35252 bench
8456 bindbench
2544 bindrequest
48672 blkid
31404 cache
2752 cat
3028 cgi
26192 chattr
9788 checkpassword.login
5428 chgrp
3556 chmod
5428 chown
2152 chroot
2936 chrootuid
1208 chvt
1028 clear
2092 cmp
16768 connect
6484 cp
4028 date
3084 dd
1944 df
5432 diet
1124 dirname
49044 dl
1540 dmesg
8548 dnsd
20864 dnswhitelist
3532 doit
1248 domainname
5864 du
10588 dumpacls
86996 dumpe2fs
7212 dumpidx
300644 e2fsck
76196 e2image
58948 e2undo
1320 echo
2408 elftrunc
3032 env
1004 false
27776 fget
14196 fget_nodns
7116 fgetty
21676 filefrag
15520 fnord
20908 fnord-cgi
17608 fnord-idx
8268 forkbench
8288 forksbench
3184 free
57008 fsck
39480 ftpd
136908 gatling
12792 genadd
11480 gencomba
8744 gen_crc32table
10184 getlinks
1516 hard-reboot
1756 hcat
2412 head
73764 heute
10728 hitprofile
14024 host
1228 hostname
24652 httpbench
16336 httpd
4404 id
7168 idx2ldif
2028 insmod
8604 install
22404 ioerr
2068 kill
1852 killall5
23124 lb
6828 lbstat
11944 ldapclient
9512 ldapclient_str
6816 ldapdelete
6508 ldd
3444 ln
2860 login1
4916 login2
16440 logsave
14716 ls
26212 lsattr
1648 lsmod
4 mailsorter -> doit
8252 manymapbench
10372 matchiprange
4388 md5password
3500 md5sum
1312 mesg
8908 minit
6396 minit-update
65872 mkcdb
3048 mkdir
191256 mke2fs
2996 mkfifo
12960 mklost+found
2996 mknod
2096 mktemp
7748 mktestdata
8204 mmapbench
8476 mount
7064 msvc
6484 mv
8000 mysql2ldif
24904 ncp
21472 netcat
2136 nice
2460 nohup
3 npoll -> ncp
3 npush -> ncp
6244 pidfilehack
14604 ping
16764 pingnet
7352 ping-nodns
1308 pivot_root
1196 printenv
13652 pthreadbench
1140 pwd
10304 record
10696 referrer
9772 rellink
1632 renice
105328 resize2fs
3488 rm
2112 rmdir
1556 rmmod
14948 rpoll
23336 rpush
8928 serdo
6396 shutdown
1748 sleep
3376 sleep2
1224 soscp
1072 sosln
1072 soslns
1072 sosmv
1120 sosrm
21124 sspeed
2768 strings
17116 subst
1012 sync
15676 syslogd
35364 t
15668 t2
6432 tail
20008 tar
19764 tcpdump
1780 tee
66096 terror
18952 terrormeter
6156 test
23000 test_extent
96808 test_icount
2736 time
4392 touch
2256 tr
15364 trafo
1000 true
1708 truncate
23628 tstat
1664 tty
174584 tune2fs
1520 umount
1448 uname
3556 uniq
2032 uudecode
1896 uuencode
16544 uuidgen
33392 vchatd
3716 wc
3680 weed
1828 which
1940 whoami
4388 write
1276 write_proc
7156 x
13200 xml
1136 yes

[Crash]

Technosaurus-

Your post is GREAT! I'm surprised there haven't been any other replies, 'cause there are all sorts of thing to play with here that deserve feedback.

Specifically, I've been trying to get a stand-alone version of mke2fs running with the Puppy Linux kernel for a couple of months now, and YOUR VERSION WORKS! Now if I can just figure out how you did it...

I've been doing some work in another post that is getting pretty stale now, but the inclusion of a working mke2fs into initrd.gz is one of the last missing pieces that I've been working on. I'll put my results in the other post as soon as I can, but in the meantime THANKS!!!

[technosaurus]

umm.... lets not mention that. I actually had to make quite a few patches for the source, but my intel video card killed that before I could do a diff. Some changes are in the includes which I managed to save but there were some changes in libblkid as well.

the standalone embutils version of the busybox apps are good middleground between busybox and the full GNU versions

I can upload my whole tree if you are interested in compiling more programs against dietlibc

This code helps a lot for compiling small static dietlibc program (leave out the -D_BSD_SOURCE unless you need it but keep the -nostdinc unless you need to remove it)

Code: Select all

CC="diet gcc -nostdinc -D_BSD_SOURCE " \
CFLAGS=" -pipe -combine -Os -ffunction-sections -fdata-sections -momit-leaf-frame-pointer -fomit-frame-pointer -mpreferred-stack-boundary=2 -fmerge-constants " \
LDFLAGS=" -Wl,-O,--gc-sections " \
./configure --prefix=/opt/diet --disable-nls --help

(--help is just a placeholder for extra flags it displays)
(the CFLAGS and LDFLAGS are selected to help optimize for size by removing duplicate functions, data, constants and pointers but can make debugging impossible so don't use for development builds)

sometimes you actually have to edit these values in the Makefiles themselves (mostly when there is no configure script)

then strip the binaries with

Code: Select all

strip --strip-all --remove-section=.comment --remove-section=.note *

[Crash]

Technosaurus,

Thanks for the information. I was in the process
of doing a source code walk-through of mke2fs
before I saw your post. Now the pressure is off
a bit, although I still want to know for instance
why Busybox claims to have it available even
though it is commented out of their code. Maybe
they had good intentions, but never could get it
to work. For such an important program, I would
think the problem had been solved multiple times
and would be readily available, but it isn't.

I'm pretty new to this Linux compiling stuff,
although my Borland C experience goes back to
version 3.1. Things are of course strange at first,
but I'm sure I'll get the hang of it. I'm going to
give diet libc a look - I've played a bit with uclibc,
with mixed success, and it never hurts to try
someone else's approach to all this.

So anyway, I made the changes to my little
project and posted it using your version of
mke2fs. Looks good, about the way I thought it
would work two months ago. I'm glad I don't do
this for a living and need to meet a deadline!

/// Edited the next morning

Come to think of it, if you posted the whole tree
it would give me a huge head start. It's probably
really big, though, and I don't know how large a
file can be put on the forum.

Another program I'm interested in is mkfsdos,
and I wouldn't mind spending some time learning
diet libc with that as an example target program.

[technosaurus]

http://puppylinux.asia/members/T/dietlibc/ puppy:linux

diet.tar.xz is the latest tree the others are a mix of my older compiles and packages from older dietlibc based distros (can't remember which)

I was even able to compile gcc-3.4.6 but have not completely bootstrapped so that I could test it completely

anyhow there are a lot of useful utils in all 3 packages especially for building an initial ramdisk - I would have put them all together but the different versions have matching names with very differing file size and functionality.

[Crash]

I downloaded the diet.tar.xz and xtradietbins.tar.xz files and uncompressed them OK. I actually had to install the XZ utilities, since I've never had to unzip a program in that format. It's always fun watching the computer work its butt off doing the make/compile, even if it is just to make the utility that you need to get to the program that you really want to use.

The second file, xdiettree.tar.xz, gives me a "403 Forbidden" error when I click on it, so there is probably a problem with its permissions.

So now I'm looking at the files, and it will probably give me countless hours of entertainment before I can accomplish something useful. But it is a great learning experience.

Again, thanks for sharing the information and the files.

[BarryK]

Found this old thread, great work by technosaurus. Have a problem, dietlibc is no longer working on Quirky Linux, now at version 8.1.3.1.
I'll send a pm to technosaurus about this post.

With the dietlibc DEB package installed, I tried a simple hello world:

#include <stdio.h>
main()
{
printf("Hello World");
}

/usr/bin/diet gcc -nostdinc helloworld.c -o helloworld

In file included from /usr/include/diet/time.h:6:0,
from /usr/include/arm-linux-gnueabihf/sys/types.h:132,
from /usr/include/diet/stdio.h:5,
from helloworld.c:1:
/usr/include/diet/signal.h:252:3: error: unknown type name ‘int32_t’
int32_t si_signo;
^

/usr/include/diet/stdio.h has a line "#include <sys/types.h>", so the
big question is, why doesn't it retrieve
/usr/include/diet/sys/types.h?

Instead, it gets /usr/include/arm-linux-gnueabihf/sys/types.h, despite
the "-nostdinc".

I changed the line in stdio.h to "#include
"/usr/include/diet/sys/types.h"", now helloworld.c compiles.

So, the big question is, why does it retrieve the correct stdio.h, but
not the correct sys/types.h?

If anyone can give me some idea why this is happening, most appreciated!
Try the hello world compile in your pup, does it work? You do of course need the "devx" pet installed, and with most pups that comes with dietlibc.

[technosaurus]

adding -I/usr/include/diet/ (or equivalent) to the gcc command or cflags may help... though I mostly use musl libc or my own barebones c library these days. The size difference between musl and dietlibc is negligible and musl is more conformant, more actively developed and has a permissive license that makes static linking less of a PITA

IIRC, when I used to do these things I would temporarily move /usr/include/ to /usr/include~ (or whatever) and symlink a separate dietlibc or uclibc include directory to /usr/include. Whenever something was missing, I would just add an appropriate symlink. I would also do similar things with the glibc compiled *.so files/symlinks in /lib and /usr/lib - putting them into /usr/local/lib~ so that they wouldn't accidentally override the dietlibc compiled *.a files

If I were doing a distro for distro builders (like T2 and buildroot), each toolchain would get its own set of directories as described above.

[BarryK]

adding -I/usr/include/diet/ (or equivalent) to the gcc command or cflags may help...

The problem here is the "-nostdinc". at die.net:
https://linux.die.net/man/1/gcc

-nostdinc
Do not search the standard system directories for header files. Only the directories you have specified with -I options (and the directory of the current file, if appropriate) are searched.

Now, if I use the "-v" verbose option for diet, as explained here:
https://linux.die.net/man/1/diet

Code: Select all

# /usr/lib/arm-linux-gnueabihf/diet/bin/diet -v gcc -nostdinc helloworld.c -o helloworld
gcc -nostdlib -static -L/usr/lib/arm-linux-gnueabihf/diet/lib-arm -L/usr/lib/diet/lib /usr/lib/arm-linux-gnueabihf/diet/lib-arm/start.o -nostdinc helloworld.c -o helloworld -isystem /usr/lib/arm-linux-gnueabihf/diet/include -D__dietlibc__ /usr/lib/arm-linux-gnueabihf/diet/lib-arm/libc.a -lgcc /usr/lib/arm-linux-gnueabihf/diet/lib-arm/libc.a

Notice the "-isystem", it is:

-isystem dir
Search dir for header files, after all directories specified by -I but before the standard system directories. Mark it as a system directory, so that it gets the same special treatment as is applied to the standard system directories. If dir begins with "=", then the "=" will be replaced by the sysroot prefix; see --sysroot and -isysroot.

Note, /usr/lib/arm-linux-gnueabihf/diet/include is a symlink to /usr/include/diet, so the path t the dietlibc headers is OK.

See the problem? gcc has been told to ignore the system headers entirely, only use those specified. However, it is not doing as it is told, and is finding /usr/include/arm-linux-gnueabihf/sys/types.h

This is totally wrong. I have sent an email to Felix, the creator of dietlibc, maybe he has encountered this problem before.

[BarryK]

Fixed it!

Have to export these environment variables, so gcc knows where the system headers are, so that "-nostdinc" knows what to ignore!

I thought that gcc would already know that, given that it is compiled by ubuntu as multilib.

Code: Select all

# export C_INCLUDE_PATH=/usr/include/arm-linux-gnueabihf
# export CPLUS_INCLUDE_PATH=/usr/include/arm-linux-gnueabihf
# 
# /usr/lib/arm-linux-gnueabihf/diet/bin/diet -v gcc -nostdinc helloworld.c -o helloworld
gcc -nostdlib -static -L/usr/lib/arm-linux-gnueabihf/diet/lib-arm -L/usr/lib/diet/lib /usr/lib/arm-linux-gnueabihf/diet/lib-arm/start.o -nostdinc helloworld.c -o helloworld -isystem /usr/lib/arm-linux-gnueabihf/diet/include -D__dietlibc__ /usr/lib/arm-linux-gnueabihf/diet/lib-arm/libc.a -lgcc /usr/lib/arm-linux-gnueabihf/diet/lib-arm/libc.a 
helloworld.c:4:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
 main()
 ^
/usr/lib/arm-linux-gnueabihf/diet/lib-arm/libc.a(vprintf.o): In function `vprintf':
(.text+0x2c): warning: warning: the printf functions add several kilobytes of bloat.
# 

[goingnuts]

embutils gives standalone programs although an effort to do a multicall binary of some of the programs has been done in the "allione"-program. I needed an embutils-box.

Attached the result, source is embutils-0.19, used dietlibc-0.28 (but newer versions works too), applied the following patches from T2: embutils-0.19-build.diff, embutils-0.19-free.diff, embutils-0.19-losetup.diff, embutils-0.19-mount.diff, embutils-0.19-no-shared.diff, embutils-0.19-which-wd.diff, embutils-0.19-x-switch_root.diff and embutils-0.19-xx-readlink.diff (renamed - and attached as well in T2_embutils.patch.gz).

On top of that the attached patch (embutils-0.19_mcb.patch.gz) is applied to get sources converted to build a multicallbinary (mcb).

Final size of mcb_embutils-0.19 is 118 K/57K upxed holding the following programs:

Code: Select all

arch basename cat chmod chown chroot chrootuid chvt clear cmp date dd df dirname dmesg domainname du echo env false free head hostname id insmod install kill ln losetup ls lsmod md5sum mesg mkdir mknod mktemp mount mv nice nohup pivot_root printenv pwd readlink renice rm rmdir rmmod sleep sleep2 soscp sosln soslns sosmv sosrm strings switch_root sync tail tar tee test time touch tr true truncate tty umount uname uniq uudecode uuencode wc which whoami write yes chgrp cp mkfifo

Most of the programs do what they should although documentation/help is little and could be improved with little/some effort.

To exercise the mcb_embutils-0.19 put it in a test directory and do ./mcb_embutils-0.19 --install.
The symlinks will be created in same directory for further testing.

To reconfigure and recompile you have to unpack original source, apply the patches from T2, apply the embutils-0.19_mcb.patch, change permissions of "config_me" and "build_me", run "config_me" and finally run "build_me".

A small test is included - "testsuite" - copy to your test directory, change permissions and run it to simple test most of the programs.

Original source link.