Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 26 Nov 2014, 11:18
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
Compiling with musl (an alternate libc)
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 5 Posts_count   Goto page: 1, 2, 3, 4, 5 Next
Author Message
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Mon 07 May 2012, 21:28    Post_subject:  Compiling with musl (an alternate libc)  

Apologies if this is too on-topic...

What is it, and why would I be interested?
musl is an alternate libc (standard C library), with:
-a stable ABI
-strict full X/Open 2008 (SUS4), POSIX (latest version), and ISO C99 conformance.
-substantial _GNU_SOURCE compatability
-C++ compatability
-for a week or two, -fstack-protector compatability (I haven't tested)
-an MIT license (changed yesterday or so from LGPL 2.1)
-size competetive with uclibc (slightly larger than dietlibc)
-modular design that makes for small static binaries
-TCB shadow passwords
-supports i386, x86_64, and arm (mips is planned)
-fairly easy to port
-responsive developer

You might be interested if you...:
-Want to build static binaries for linux on supported architectures
-want a small libc that doesn't break the abi regularly (vs uclibc)
-want full standard conformance, rather than "we pick how much of the standard we follow"

Building it:
There are two options: as a cross-compiler, or migrating the system.
If you migrate, you'll want to read the documentation included. I can supply some details, but that's for a later post.

1. Compile and install musl, preferably from git head (a release is a tag, not a branch, so all fixes are in git).
Code:
 git clone git://git.etalabs.net/musl

-Copy config.mak from dist/ and adjust as you want.
You may want to disable the shared library.
(PREFIX defaults to /usr/local/musl, and should not be set to /, /usr, or /usr/local without following the documentation on migrating)

2. Get the linux kernel headers (linux-libc-dev, not the ones for building modules) and put them somewhere (I use /opt/musl/lin)
There should be a linux/ subdirectory in the directory you install the headers in.

3. export CC=musl-gcc; export CFLAGS="-Os -static -fno-stack-protector -I/opt/musl/lin -D_GNU_SOURCE"
Don't forget the -D_GNU_SOURCE - most stuff won't build without that, though some will. -I/... should point to the linux headers.
(Note-the linux headers are needed for OSS and a lot besides)

4. Now run configure and make.


If you want to discuss anything related to musl, feel free to use this thread.
Back to top
View user's profile Send_private_message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Tue 08 May 2012, 10:30    Post_subject:  

Bookmarked for further reference.
This should be under "Cutting Edge" - interesting stuff here Very Happy
I haven't tried musl, but I suppose it will compile busybox just fine?

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Tue 08 May 2012, 14:18    Post_subject:  

jamesbond wrote:
Bookmarked for further reference.
This should be under "Cutting Edge" - interesting stuff here Very Happy
I haven't tried musl, but I suppose it will compile busybox just fine?


Much of Busybox.
You must install the kernel headers and point it at them (-I/opt/musl/lin)
Also, you need to disable NFS, unless you've built tirpc (RPC support is mandatory for NFS). I remember some other stuff I turned off, too...

Busybox won't build without -D_GNU_SOURCE, IIRC. You might also need -Wl,-z,muldefs -- but probably won't with any recent musl release.

Long story short: I got a static busybox binary that does anything most people need, in ~700 kilobytes. That includes vi, ash, init, and mdev, so you can boot to a shell with no other binaries.

I've also built gcc 3.4 (there were some breakages in gcc 4.x), make 3.81, ncurses 5.9, vim 7.3, m4 from netbsd 4.0, zlib, wireless-tools 30pre9 (which is harder to do...), and a few other packages.
Back to top
View user's profile Send_private_message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Wed 09 May 2012, 22:22    Post_subject:  

Impressive Very Happy

This is my attempt at building dnsmasq as 32-bit executable under 64-bit environment.

1. I installed musl (0.9.0) - I added "-m32" to config.mak as directed. Everything went well. The build is *much much* faster than building uclibc using buildroot.
2. Unpacked dnsmasq, and did the export as indicated in the first post - fail. Looking at the output, I see it called gcc instead musl-gcc - so I passed CC and CFLAGS through make instead:
Code:
make CC=musl-gcc  CFLAGS"=-Os -static -fno-stack-protector -D_GNU_SOURCE -m32 -I /mnt/sda2/tmp/linux-3.2.2/usr/include/"

3. Failed - missing netinet/in_systm.h. I checked, /opt/musl/include/netinet doesn't have in_systm.h. So I concocted my own from glibc, suitably modified:
Code:
#ifndef _NETINET_IN_SYSTM_H
#define _NETINET_IN_SYSTM_H 1
typedef uint16_t n_short;      /* short as received from the net */
typedef uint32_t n_long;       /* long as received from the net  */
typedef uint32_t n_time;       /* ms since 00:00 GMT, byte rev   */
#endif /* netinet/in_systm.h */

4. Went smoothly until it tried to compile log.c, then failed - cannot find _PATH_LOG. This is defined in bits/syslog-path.h, but /opt/musl/include/bits doesn't have this. I just add _PATH_LOG to dnsmasq's config.h and compilation continued and finished.
5. But it failed at link step - saying that it skipped libc.a when searching -lc. This would not happened if I compiled for 64-bits, so I added LDFLAGS=-m32 to the make file, which now looks like this:
Code:
make CC=musl-gcc  CFLAGS"=-Os -static -fno-stack-protector -D_GNU_SOURCE -m32 -I /mnt/sda2/tmp/linux-3.2.2/usr/include/" LDFLAGS=-m32

6. All went well - except that ld cannot find capset/capget.
Code:
dnsmasq.c:(.text+0x1172): undefined reference to `capget'
dnsmasq.c:(.text+0x14d4): undefined reference to `capset'
dnsmasq.c:(.text+0x156a): undefined reference to `capset'

This is where I stopped. grep -Ril capset /opt/musl/* told me there is a definition of capset in bits/syscall.h, but there is none in the libs.

EDIT:
7. I edited dnsmasq.c to remove all references to capset and capget, and the compile ended successfully. I ended up with 172k stripped dnsmasq static 32-bit binary that runs successfully within my 64-bit system - but now I need to test it that my removal of capset/capget doesn't upset it too much Smile. The size compares favorably, my stripped *dynamic* 64-bit dnsmasq is 176k Shocked

EDIT: It works.

This capset/capget thing seems to be linux specific, and the POSIX version of the same stuff is implemented in libcap, however it can also be found in glibc (at least it seems to, I may be wrong).

Questions:
1. I'm sure the above is not the only one, so the general question is: as I understand it musl strives to be POSIX-compliant (not Linux compliant), however it also strives to be a glibc replacement. musl already implements _GNU_SOURCE, what about compatibilities at this level? Unfortunately, compatibility support == bloat Confused

2. Assuming:
- not trying to cross-compile (not as above)
- not trying to migrate the whole system
- building a static binary
How does libs compiled with musl co-operate with libs compiled with glibc? Let's say I already have a libxxxx compiled with glibc (let say openssl's libssl). Will this work with musl - i.e. if I use musl-gcc and include does -lssl (pointing to the appropriate -L) - will this work? Probably not, I suppose.
If not, then I need to compile openssl again and install it on, say, /opt/musl/openssl, and make sure that all my compiles use that version of libgtk+, no?

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Thu 10 May 2012, 14:47    Post_subject:  

jamesbond wrote:
...
2. Unpacked dnsmasq, and did the export as indicated in the first post - fail. Looking at the output, I see it called gcc instead musl-gcc - so I passed CC and CFLAGS through make instead:
Code:
make CC=musl-gcc  CFLAGS"=-Os -static -fno-stack-protector -D_GNU_SOURCE -m32 -I /mnt/sda2/tmp/linux-3.2.2/usr/include/"

Are those the cleaned headers corresponding to linux-libc-dev, or the linux kernel headers?
Quote:

3. Failed - missing netinet/in_systm.h. I checked, /opt/musl/include/netinet doesn't have in_systm.h. So I concocted my own from glibc, suitably modified:
Code:
#ifndef _NETINET_IN_SYSTM_H
#define _NETINET_IN_SYSTM_H 1
typedef uint16_t n_short;      /* short as received from the net */
typedef uint32_t n_long;       /* long as received from the net  */
typedef uint32_t n_time;       /* ms since 00:00 GMT, byte rev   */
#endif /* netinet/in_systm.h */

4. Went smoothly until it tried to compile log.c, then failed - cannot find _PATH_LOG. This is defined in bits/syslog-path.h, but /opt/musl/include/bits doesn't have this. I just add _PATH_LOG to dnsmasq's config.h and compilation continued and finished.
...
6. All went well - except that ld cannot find capset/capget.
Code:
dnsmasq.c:(.text+0x1172): undefined reference to `capget'
dnsmasq.c:(.text+0x14d4): undefined reference to `capset'
dnsmasq.c:(.text+0x156a): undefined reference to `capset'

This is where I stopped. grep -Ril capset /opt/musl/* told me there is a definition of capset in bits/syscall.h, but there is none in the libs.

EDIT:
7. I edited dnsmasq.c to remove all references to capset and capget, and the compile ended successfully. I ended up with 172k stripped dnsmasq static 32-bit binary that runs successfully within my 64-bit system - but now I need to test it that my removal of capset/capget doesn't upset it too much Smile.
Questions:
1. musl already implements _GNU_SOURCE, what about compatibilities at this level? Unfortunately, compatibility support == bloat Confused

2. Assuming:
- not trying to cross-compile (not as above)
- not trying to migrate the whole system
- building a static binary
How does libs compiled with musl co-operate with libs compiled with glibc? Let's say I already have a libxxxx compiled with glibc (let say openssl's libssl). Will this work with musl - i.e. if I use musl-gcc and include does -lssl (pointing to the appropriate -L) - will this work? Probably not, I suppose.
If not, then I need to compile openssl again and install it on, say, /opt/musl/openssl, and make sure that all my compiles use that version of libgtk+, no?

Removing capset/capget is most likely to be a security issue--they allow checking and dropping permissions in a more grnular way than setuid/seteuid.
Don't use grep -Ril --you will be very thoroughly fooled, until you actually look at the file. Here it's only SYS_capset and _NR_capset that are defined.
Currently musl is NOT ABI compatible with glibc; you will have to recompile everything. Substantial chunks of glibc headers are nonstandard, and aren't defined in musl yet.

If you report a missing function that breaks some program to the musl mailing list (refer to http://www.etalabs.net/musl/community.html), it may well be added, occasionally even the same day. This is what I mean by "responsive". I'd use a subject like
"Compatability: cap{set,get} needed for dnsmasq"
(identify that the issue is compatability, name functions, and name the program you're trying to build.)
The goal seems to be "enough glibc compatability for most software to work".

The "appropriate" -L is likely to be -L/usr/include, which will massively break everything.
I presume the "libgtk+" was a typo?
You can install in any prefix where there aren't glibc-based libraries.
In other words, if the musl prefix is /opt/musl, you can use that as the prefix for all the libraries, and musl-gcc will find them all.

HTH
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Thu 10 May 2012, 16:36    Post_subject:  

I've confirmed this, sent an email to the musl mail-list with patches for the first two issues (missing header, add _PATH_LOG to syslog.h when _(GNU|BSD)_SOURCE is defined)
And the patches are committed in the bsd branch of my repo (github.com/idunham/musl).
Back to top
View user's profile Send_private_message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Thu 10 May 2012, 19:21    Post_subject:  

Ibidem wrote:
Are those the cleaned headers corresponding to linux-libc-dev, or the linux kernel headers?

cleaned headers corresponding to linux-libc-dev. I did try with raw kernel headers - didn't work, of course Embarassed


Quote:
Removing capset/capget is most likely to be a security issue--they allow checking and dropping permissions in a more grnular way than setuid/seteuid.
Yes.

Quote:
Don't use grep -Ril --you will be very thoroughly fooled, until you actually look at the file. Here it's only SYS_capset and _NR_capset that are defined.
That's just a quick way to check stuff; if grep -Ril returns negative the the symbol is definitely not there. But you are right; I did check the file afterward.

Quote:
Currently musl is NOT ABI compatible with glibc; you will have to recompile everything. Substantial chunks of glibc headers are nonstandard, and aren't defined in musl yet.
Yes, I must not be thinking when I asked this question. When libs are compiled they include constants and symbols and pointer offsets etc that come from the libc; and these will be different between glibc and musl. So everything must be re-compiled (and stored elsewhere) even if all I want to do is making a static binary out of it.

Quote:
If you report a missing function that breaks some program to the musl mailing list (refer to http://www.etalabs.net/musl/community.html), it may well be added, occasionally even the same day. This is what I mean by "responsive". I'd use a subject like
"Compatability: cap{set,get} needed for dnsmasq"
(identify that the issue is compatability, name functions, and name the program you're trying to build.)
The goal seems to be "enough glibc compatability for most software to work".
Superb ! Very Happy

Quote:
The "appropriate" -L is likely to be -L/usr/include, which will massively break everything.
Which I tried and as you said - breaks everything (didn't even compile).

Quote:
I presume the "libgtk+" was a typo?
Yes Embarassed - was using libgtk+ for example, then changed my mind and change to libssl, but obviously I didn't replace all the references. Embarassed

Quote:
You can install in any prefix where there aren't glibc-based libraries.
In other words, if the musl prefix is /opt/musl, you can use that as the prefix for all the libraries, and musl-gcc will find them all.
Thank you, this is what I need to know. I want to avoid appending extra -L and -I unless it's really necessary. I suppose this will also work with packages using pkg-config? I probably have to append /opt/musl to PKG_CONFIG_PATH?

I will try to compile a few others and see how far I get.

EDIT: removed comment about in_systm.h, I see you have raised that on the mailing list too.

cheers!

_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Sat 12 May 2012, 03:12    Post_subject:  

pkg-config seems designed to break cross-compiles.
if you use it, any packages missing for the cross-compiler and present for the host will get picked up...again, instant breakage.

I guess you need PKG_CONFIG_LIBDIR; set it to the directory containing .pc files in /opt/musl (probably /opt/musl/lib/pkgconfig, but some packages use oddball directories...)

capset & capget apparently belong in libcap; it seems glibc supports the symbols for backwards compatibility, and hasn't had it in headers since 1999 or so.. Rich doesn't think much of the "capabilities" approach, says it's mostly a false sense of security.

_PATH_LOG is used for some insane workaround for broken syslog daemons; looks like there should be an easy patch, though (change the SUN/ANDROID check to add !defined(_PATH_LOG).
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Sat 12 May 2012, 16:48    Post_subject:  

I'm considering building a musl snapshot "SDK" of sorts.
Basically, it would have whatever's needed to start building against musl, installed in /opt/musl-09 .
Right now I think these would be what I'd start with:

musl
linux-libc-dev
ncurses (pdcurses is also an alternative, but I don't see a reason)
zlib
openssl
libcurl
libbz2
liblzma
libcap
libtirpc
libnl3

perl would be provided by the host, as would gcc and binutils (though I could just use pcc...); all parts except the Linux headers (which Torvalds appears to consider PD once sanitized) and libnl3 (LGPL) are PD, MIT, or BSD licensed, so strictly speaking don't require source (though I'd have that available).
But, OTOH, I'd be publishing the build-scripts and any patches...
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Mon 14 May 2012, 20:22    Post_subject:  

Ibidem wrote:

musl
linux-libc-dev
ncurses
zlib
openssl
libcurl
libbz2
liblzma
libcap
libtirpc
libnl3

libssh2 builds OOB with openssl and zlib installed
libnl3, zlib, ncurses, and openssl build with minimal patching
libcap needs pam
pam needs hacking, and still doesn't quite build
pam wants RPC, which means libtirpc...(dependency can be hacked out)
libtirpc is probably a porter's worst nightmare...
Back to top
View user's profile Send_private_message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Mon 14 May 2012, 21:01    Post_subject:  

Thanks Ibidem, this is cool! I could get the SDK from your git repo I suppose?
_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Tue 15 May 2012, 02:22    Post_subject:  

I presume you mean github.com/idunham/musl ?
Probably not that one--I'm thinking of something more like /usr/src is on certain systems.
*If* I do an SDK, it would have a repo for the build environment, and then I'd have a binary download also available from that repo.
There's no good reason to build a musl libc binary package, without an environment--it takes maybe 2 minutes to build on an Atom, you need updates and bugfixes, and so on.
But if I distribute libnl3/pam, I need to distribute sources anyhow...
Back to top
View user's profile Send_private_message 
jamesbond

Joined: 26 Feb 2007
Posts: 2230
Location: The Blue Marble

PostPosted: Tue 15 May 2012, 07:18    Post_subject:  

I was thinking along you'd post a build script on github.com/idunham/musl that will download source packages (the ones you've listed) and relevant patches to make it compile with musl; and compile it to some location (/opt/musl-09 is good) Very Happy
_________________
Fatdog64, Slacko and Puppeee user. Puppy user since 2.13.
Contributed Fatdog64 packages thread
Back to top
View user's profile Send_private_message 
Flash
Official Dog Handler


Joined: 04 May 2005
Posts: 11164
Location: Arizona USA

PostPosted: Tue 15 May 2012, 08:39    Post_subject:  

You guys feel free to invite the developers to join our forum. Explain to them how useful Puppy could be to a developer. Smile
Back to top
View user's profile Send_private_message 
Ibidem

Joined: 25 May 2010
Posts: 523
Location: State of Jefferson

PostPosted: Tue 15 May 2012, 10:58    Post_subject:  

Flash wrote:
You guys feel free to invite the developers to join our forum. Explain to them how useful Puppy could be to a developer. Smile

Most of them already have their own projects/distros Wink...
I'm aware of about 20 people using it, though there may be more.
I also know of at least 3 small distros that use it (jhuntwork's LightOS, Sabotage Linux which has as many git repos as users, bootstrap-linux which may not be commonly used, but can recompile itself with only 6 packages)
Openwall GNU/*/Linux is going to switch to musl at some point.

musl has an IRC channel (#musl on freenode) and a mailing list.
Also, I'm not sure the Puppy way/philosophy is acceptable to many of them...

Somehow, I doubt that the invitation would go over too well.
Back to top
View user's profile Send_private_message 
Display_posts:   Sort by:   
Page 1 of 5 Posts_count   Goto page: 1, 2, 3, 4, 5 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Off-Topic Area » Programming
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1267s ][ Queries: 13 (0.0050s) ][ GZIP on ]