Puppy From Scratch, Beyond Puppy From Scratch

Under development: PCMCIA, wireless, etc.
Post Reply
Message
Author
User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

Puppy From Scratch, Beyond Puppy From Scratch

#1 Post by technosaurus »

I have made the "easy" decision that I want to build a Puppy From Scratch, but there are a couple of things that I would like to try out and the harder decision is which to do....

This is mainly to test some of the new features of gcc-4.5 & gold - specifically -flto and -fwhopr in comparison with -ffunction-sections, -fdata-sections + --gc-sections (both size and speed)

Static uclibc Puppy - where all binaries are static
Mostly static Puppy (glibc) - only the main toolkits (gtk, qt...) and their dependencies are shared - all other libs are static.
Arm Puppy - standard puppy using my pepperpad2 as a test platform.
Commercial Puppy - Static libraries where the license permits, to ease dependency and functionality problems across distros.

Static uclibc Puppy
downsides - lots of patching, proprietary software like flashplugin wouldn't work
upsides - binaries would work in any distro for a long time
(many other well documented +/- of static binaries exist besides these)

Mostly static Puppy
downsides - possibly size depending on package selection
(other static +/- to some extent)
upsides - speed boost of static linking combined with LDPRELOAD (via .xinitrc) for all programs & proprietary software will still work

Arm Puppy
downsides - untested waters for me and probably a significant amount of patching
upsides - $99 Puppy laptops and tablets

Commercial Puppy
downsides - invasion of the leaches, support, legal...
upsides - more available software in the future?

I am leaning toward Mostly Static Puppy, because I have done some experimenting in that direction already... with really good results. If someone has a persuasive argument for one of the other options though I would consider (otherwise I wouldn't have mentioned them)

On a separate note, the developer of Magic Ermine granted me a license so that I can package any open source software to be run in a single executable even if it requires additional libraries or files from /etc or /usr/share .... basically a portableapps for linux - the static or mixed static method would help keep the sizes smaller (since the alternative is to include each entire shared library or tell it that it will be external and thus a "dependency")
Last edited by technosaurus on Wed 14 Jul 2010, 23:55, edited 1 time in total.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#2 Post by technosaurus »

<mostly static process>
toolkit library is dynamically linked with other dynamic libraries using -fpic (NOT -fPIC) = better speed and smaller size

libraries that are known to not be linked against directly (or at least not often) are directly linked into the respective toolkit (compiled with --enable-static --disable-shared --with-pic and various CFLAGS/LDFLAGS)

additional "static libraries" (object archives) that won't be linked into shared libraries will be compiled for inclusion in binaries only (with --enable-static --disable-shared --without-pic and various CFLAGS/LDFLAGS)

use LDPRELOAD for gtk2 in .xinitrc prior to starting ROX (it loads anyways when you run rox, but this should make rox and others start faster)

The secondary purpose is to examine the differences between new features in gcc-4.5 (-flto and -fwhopr link time optimizations) and the older *sections flags as well as combinations of them.

example:
libX11 has a dependency on libXau and libxdmcp, but almost nothing ever links directly with them, so compile Xau and xdmcp with --enable-static --disable-shared --with-pic and libX11 will no longer have them as dependencies. The size of libX11 only goes up slightly (not near the size of Xau and xdmcp), but will load faster because there are less libs to look for and less relocations. (Next recompile Xau and/or xdmcp without pic in case a later program calls for them directly)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#3 Post by Iguleder »

I think it's a good idea, but there's something you need to consider: GCC's Graphite loop optimizations. They made my distro about much bigger, but things are supposed to be more optimized. That requires GCC to be built against ppl-cloog and ppl. I did two builds of my whole distro, with and without, the difference is about 25-30%.

By the way, Mozilla spotted big regressions with GCC 4.5.0 ... you should wait for 4.5.1 or drop to the 4.4.x series. Most major distros these days are still on 4.4.x.

EDIT: don't even try uClibc. Things just don't work with it.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#4 Post by technosaurus »

Iguleder wrote:I think it's a good idea, but there's something you need to consider: GCC's Graphite loop optimizations. They made my distro about much bigger, but things are supposed to be more optimized. That requires GCC to be built against ppl-cloog and ppl. I did two builds of my whole distro, with and without, the difference is about 25-30%.

By the way, Mozilla spotted big regressions with GCC 4.5.0 ... you should wait for 4.5.1 or drop to the 4.4.x series. Most major distros these days are still on 4.4.x.
Such is the case with all X.0 releases of gcc, but someone has to report the bugs so that they can be worked out. If you are including the static libraries in your size, that is why... gcc ADDs code for link time optimization AND gold is required to take advantage of it (at least for now) I still find the gcc-3.X series superior in many areas and would minimally want a X.x.3 release for a stable, production distro, but I may try llvm when llvm-2.8 is released. Also as to the sizes being larger - it sounds like a gcc lib could be getting linked in statically (I have seen some kind of patch for this, and it has been a while, but I think it was by linuxfromscratch or one of the build systems)
Iguleder wrote:EDIT: don't even try uClibc. Things just don't work with it.
That's what they say about dietlibc too, but I had a whole diet distro going before I started trying to verify license compatibilities - getting it "working" is only the second hardest part. eglibc should suffice though for most purposes

p.s. T2-8.0 was just released.
Last edited by technosaurus on Thu 15 Jul 2010, 14:40, edited 1 time in total.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

tubby
Posts: 317
Joined: Sat 24 Jan 2009, 15:49

#5 Post by tubby »

I would like to see you go along the ARM route, BigBass has now released his latest slack model which looks like a good starting point for the puppy arm system.
By the way i am being totally selfish as i have one of those sub £100 netbooks :) and i am in the progress of trying to port puppy linux myself.

User avatar
jemimah
Posts: 4307
Joined: Wed 26 Aug 2009, 19:56
Location: Tampa, FL
Contact:

#6 Post by jemimah »

Hmm, Ttuuxxx and I have been discussing a puppy from scratch project too. Perhaps a combination of forces would make a powerful triumvirate.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#7 Post by Iguleder »

I completed a build of the base for a "Puppy from Scratch" yesterday, with all base packages and most libraries, but without X. It's about 570 MB with tons of console applications plus everything for the devx, including the full Perl and Python.

It's GCC 4.4.4, though, because 4.5.0 produces large binaries and some say it's buggy ... plus, it has one extra dependency that can be left out.

I just don't have any idea how to make a Puppy from this thing ... that's the main problem. :(
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#8 Post by technosaurus »

ftp://gcc.gnu.org/pub/gcc/snapshots/4.5.1-RC-20100722/ may fix issues?
... the new gimple data may require different strip utilities and/or parameters to obtain smaller sizes ... how do nm and objdump runs on binaries compare for each section
Also dependencies are not as important for devx utils
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
DaveS
Posts: 3685
Joined: Thu 09 Oct 2008, 16:01
Location: UK

#9 Post by DaveS »

Where do you guys find the time............. :)
Spup Frugal HD and USB
Root forever!

User avatar
vtpup
Posts: 1420
Joined: Thu 16 Oct 2008, 01:42
Location: Republic of Vermont
Contact:

#10 Post by vtpup »

Well, I'd be super happy to see an ARM 4 puppy, not only for an antidote to the Ipad by applying it to touchscreen tablets like the Eken M001, but because sub $100 laptops and netbooks are what many people need to compute at all. These inexpensive laptops and tablets all have limited processing and memory ability, and would be greatly benefited by Puppy, therefore. Personally, I think something like that helps the world.

Many run Android or Win CE, both of which have many blocks to personal freedom, privacy, and the ability to develop or understand the machine's inner workings. They are designed to remove capability, and enforce consumerism and dependency. Puppy is not only the most effective OS, I believe, for these, but also the most transparent and educational.

Frankly I can think of no more appropriate currently manufactured platform for a version of Puppy than these low cost computers. ARM 4 would include most of them and many other devices, including phones, PNAs/GPS, etc for compatibility. This in turn opens the door for development of more sophisticated Linux/open source mapping apps, etc.

I have currently set up a small contributors bounty for development of Debian compatible touchscreen drivers for the Eken M001 through the Slatedroid forum, but I'd love to see Puppy even more than Debian running on one.

The main reason for the Deb bounty is that Debian has already been successfully ported to ARM 4 and is now running on the Eken M001 (with an external USB keyboard and mouse). Touchscreen drivers are the only block at this point to an out of the box Debian M001 tablet. Puppy would be even more desirable, and almost certainly, better performing.

gortonc
Posts: 39
Joined: Tue 12 Jan 2010, 16:05
Location: Costa Rica

#11 Post by gortonc »

What he said!
I am with vtpup.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#12 Post by technosaurus »

just narrowing down the infrastructure plan a bit:

try to compile most gtk2 dependencies as static libs with position independent code and the link them all into one shared library with symlinks for all of the dependencies ... thus allowing garbage collection across all sections (compiled with -ffunction-sections -fdata-sections and --gc-sections). This will result in a larger gtk-x11-2.0.so.* library but decrease the overall size by up to 50% and will require changes to .pc files, symlinks and the --as-needed flag for binaries but greatly benefit from LDPRELOAD of the gtk2 library during the init process (This will also counter the excess baggage for programs that only need one of gtk2's dependencies)

Possibility exists to do this for qt and others, possibly even tiered atop gtk2, for example: webkitgtk with libsoup, gnutls and gnupg combined with only gtk2 dependency.

Why do it this way when the standard/traditional way works??? I don't like waiting until after I have clicked on an application for it to start. I want it to already be 90% loaded and completely startup before I have even released the mouse button. Maybe one of these days AI will improve enough that I won't even have to click it.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Post Reply