pUPnGO - 6Mb ISO - Basic Building Block Puplet

[technosaurus]

Your welcome.
I currently have jwm, rxvt, getgui, xfm and started on xpaint with plans for xedit, mupdf and various panel applets. I am using wary with its toolchain & shared libs though for initial testing, so it may need patching for uclibc (or musl libc) and tinyx11 still. We will see how it goes.

[PANZERKOPF]

Your welcome.
I currently have jwm, rxvt, getgui, xfm and started on xpaint with plans for xedit, mupdf and various panel applets.

Xfm is great FM but latest one has a bug (cannot open directory). Old Xfm hasn't that
bug but mouse wheel is unsupported. (this based on my personal experience. Maybe I am wrong?)
Because we like tiny and fast tools, let me ask:
Why not gtk1?
It is fast and small but unfortunately forgotten toolkit. We have lot of apps based on it, even new versions still support it (like MtPaint, MPlayer). IMHO it is better than Athena.
I have build a small desktop system based on Xlibs and Gtk1 - runs very fast even
on very old hardware.

[technosaurus]

I've already gone there (Xpaint bumped up the size ~= gtk1+deps), the Xaw based tools end up implementing the gtk functionalities in their own way and it gets bloated fast.
I have been playing with Endeavor Mark II (gtk1 based) and wow! lots of potential ... once I finally got it going. It has quite a few builtin tools (file manager, image viewer with slideshow, archiver, recycle bin, hex editor, system monitor, dialog, splash tool, ... and _good_ documentation to boot) I didn't realize it had so much or I probably wouldn't have even bothered. I will probably settle on a choice of emelfm, gentoo or xfm instead, but I am going to archive a copy for posterity.

text editor - minimum profit 3.x
mail - sylpheed 1.x
browser - dillo-0.8.6x
paint - mtpaint
and more from here:
http://wiki.debian.org/GTK+_1.2_leftovers

BTW do you have a patch collection for the ones you have built?

[goingnuts]

Tried to make pupngo a little smaller: Created a multicall bin containing the below apps + added an info-text and installer to the main.c:

Code: Select all

#include <unistd.h>
int symlink(const char *path1, const char *path2);
 if(!strcmp(basename(argv[0]), "initrd_bundle") && (argc == 1)) {
  	fputs("Multicall binary containing:\n", stdout);
  	fputs("disktype, elspci, fusermount, find\n", stdout);
  	fputs("setserial, hotplug2stdout, cddetect\n", stdout);
  	fputs("hotplug2stdout_notimeout\n",stdout);
 }
  if (!strcmp(basename(argv[0]), "initrd_bundle") && (argc > 1)) {
    fputs("Installing symlinks...\n",stdout);
    symlink("/bin/initrd_bundle", "/bin/disktype");
    symlink("/bin/initrd_bundle", "/bin/elspci");
    symlink("/bin/initrd_bundle", "/bin/fusermount");
    symlink("/bin/initrd_bundle", "/usr/bin/find");
    symlink("/bin/initrd_bundle", "/bin/setserial");
    symlink("/bin/initrd_bundle", "/sbin/hotplug2stdout");
    symlink("/bin/initrd_bundle", "/sbin/hotplug2stdout_notimeout");
    symlink("/bin/initrd_bundle", "/sbin/cddetect");
   }
 

Run "initrd_bundle install" as first thing in init.
I know some of this should be included via BB but...

Reduced the size from 226K to 137K - oh yeah! But just to get an estimate of the potential. Its a 40% reduction and if the more heavy stuff like ntfs-3g and e2fsck could be included too its worth the huge amount of time used to make it work...
What to do with multiple defs if they reside in a local lib needed to make an app (ex. merging fileutils and findutils both need their own version of getop.c)?

[PANZERKOPF]

text editor - minimum profit 3.x
mail - sylpheed 1.x
browser - dillo-0.8.6x
paint - mtpaint

For my lightweight desktop I am using:
WM - JWM
filemanager - ROX-Filer, MC.
texteditor - Beaver (or) and Gtkedit.
mail - Sylpheed 1.x
browser - Links and Opera (Newest Opera needs only Xlibs, no Qt or other toolkits)
Paint - mtpaint
Mediaplayer - MPlayer with gtk1 GUI enabled.
Audiomixer - Gamix.
PDF Viewer - XPDF
Image Viewer - GtkSee.
Also I have created several frontends and wrappers for viewing DOC,XLS,CHM, DjVu, Etc.... files.

BTW do you have a patch collection for the ones you have built?

I borrowed sources and patches from following storages:
Barry's site http://bkhome.org/sources (Kernels, Live scripts, Xorg,.....).
Amigo's site http://amigolinux.org (GTK1 Apps).
MPlayer and BusyBox are taken from their official sites.
Base system is built from Slackware sources.
P.S.
Sorry, my english is not perfect - maybe some my explanations are difficult
for understanding.

[technosaurus]

Run "initrd_bundle install" as first thing in init.
I know some of this should be included via BB but...

Reduced the size from 226K to 137K - oh yeah! But just to get an estimate of the potential. Its a 40% reduction and if the more heavy stuff like ntfs-3g and e2fsck could be included too its worth the huge amount of time used to make it work...
What to do with multiple defs if they reside in a local lib needed to make an app (ex. merging fileutils and findutils both need their own version of getop.c)?

you can either make them compatible by modifying one or both, or just rename one and the references to it

Re busybox apps
elspci -> lspci (with some script mods)
disktype (from guessfstype?) there was a patch for blkid that does the same thing, but it wasn't mainlined
find -> find (depending on if you can mod the script that needs it)
(i do have busybox patches for guessfstype and olde2fsprogs as well as minimp3)

[faifpuhp]

This is really awesome, I'm glad it's still around (first post from 2010?)

[technosaurus]

also if you become familiar with /proc and /sys a lot of tools can be replaced with scripts
this is my eject script

Code: Select all

#!/bin/sh
echo "1" > /proc/sys/dev/cdrom/autoeject
umount /dev/cdrom

uname -a == read A < /proc/version && echo $A
(string manipulation can get the other options)

ps needs looking at /proc/[0-9]*/status and each corresponding /proc/[0-9]*/cmdline

cpuinfo == cat /proc/cpuinfo

free ~= cat /proc/meminfo

lsmod == cat /proc/modules

lspci ~~ cat /proc/bus/pci

lsusb ~~ cat /proc/bus/usb

mount with no parameters (list mounts) is ~~ cat /proc/mounts

and rather than using cat which is external to some shells and slower
read LINE < /proc/file ($LINE will be the first line)

while read LINE || [ "$LINE" ]; do echo $LINE; done < /some/file
(or you can replace echo with parsing, commands, etc)

.
.
.
.
I'm not saying C and executables are slower than a script, just that it is sometimes faster to use a builtin than to even call up an external command ... If you were inside of a C program the same would hold true ... hmm I wonder, has anyone written an init script in C? I thought about it (which is where waitpids camefrom) but ... I find myself reinventing the wheel a lot (granted, a wheel with less rotating mass usually).

[goingnuts]

This is really awesome, I'm glad it's still around (first post from 2010?)

Thanks! Me too...

I find myself reinventing the wheel a lot (granted, a wheel with less rotating mass usually).

If no one did, what you do, wheels would still be made of stone.
Further tests done: Now have disktype elspci fusermount guess_fstype setserial hotplug2stdout cddetect hotplug2stdout_notimeout mount-FULL umount-FULL losetup(-FULL) puppyinputdetect unsquashfs3.4 mksquashfs3.4 in place saving more than 50% of total size - and more to come.
Had to shift to uclibc (from dietlibc) but the potential of applications to compile static is larger (although resulting bins are also larger..)
Your hints on using /proc and /sys info are good - elimination is by far more effective than reducing size of unneeded bins...
Any thoughts on kernel-module size-reduction? Or kernel-size reduction?

[PANZERKOPF]

disktype (from guessfstype?) there was a patch for blkid that does the same thing, but

Busybox blkid shows only LABEL and UUID fields, whereas "full" blkid shows also TYPE field. Will be shown that field after applying a patch?

Or kernel-size reduction?

Sometimes I think about 2.4 version of kernel...
Version 2.4 is smaller (and maybe faster) but hasn't many drivers for newest hardware.

[technosaurus]

re: kernel size / module size

you can add a "-s" and/or rip out the "-g" to CFLAGS when building the kernel (this will strip all of the .o objects as they are made ... but I think this may happen in the kernel already with an option)

Denys Vlasenko @ busybox has patches to allow --gc-sections linking, but my experience has shown that this is incompatible with modules/plugins (but DV is clever and maybe it works, ... I haven't looked closely)

some kernel modules can be stripped with:
strip [--strip-|unneeded|debug] -R .note -R .comment *.ko
(it doesn't _always_ work, so you may need a blacklist for the unneeded/debug parts )

busybox modutils allows xz compression ... but really using compressed files in a compressed file system is usually a bad trade ... better to use squashxz and leave them alone, this will allow the squash compressor to find common strings among them ... like kprintf etc...

_never_ upx busybox (or other binaries where speed is important) , the speed penalties aren't worth it and again the size savings on a compressed file system are negligible at best and often negative

to compress an initrd with xz (kernel must support ... needs 2.6.28+, or a patch)
cd /path_to/init_tree
find . | cpio -o -H newc | xz -ze9 >../initrd.xz
(it may be easier to have a .gz extension though unless you feel like editing a bunch of stuff)
Edit- you can use an xz compressed init with a .gz extension the kernel doesn't care about the extension name, but scripts do and may specifically look for initrd.gz

but there is actually a better way - the kernel will let you specify a path to the initramfs so that it gets built in to the kernel image
(you can specify a precompressed image to but then it has to be uncompressed twice for no real benefit)
if you need to update it you can either keep the kernel around and rebuild or use an initrd.gz that will overlay (overwrite) the existing internal files
(I used to debug/improv/test the init scripts, by loading a second initrd image called myinit.gz which only contained the modified script)
If you distribute with the initramfs built in, it would be a good idea to still distribute an unused initrd.gz that just has a README

as far as kernel options go, you can now use the ext4 code for ext2 and ext3
as you have already done, eliminate modules that are <almost> always used by including them in the kernel image
if you can get away with only reading ntfs, use the kernel's and load ntfs-3g only when write access is needed
Edit

Code: Select all

#kernel params worth testing out
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_KERNEL_XZ=y
CONFIG_DEVTMPFS=y

CONFIG_DEVTMPFS_MOUNT=y
#or use busybox mount? need to test

EXTRA_FIRMWARE="space.bin separate.bin list.bin of.bin firmware.bin"
#useful for the firmware you would have in the initramfs

CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="/mnt/sda1/initramfs"
#I keep my initramfs there, change to where you have yours

http://cateee.net/lkddb/web-lkddb/DEVTMPFS_MOUNT.html
and http://cateee.net/lkddb/web-lkddb/DEVTMPFS.html
http://cateee.net/lkddb/web-lkddb/EXTRA_FIRMWARE.html
see also http://cateee.net/lkddb/web-lkddb/FIRMW ... ERNEL.html

it would be really cool if the bulk of the init process could happen in an rxvt terminal from within jwm (+Xvesa)
at a minimum you would need to enable devtmpfs (this would also allow us to drop some of the static /dev nodes) because rxvt needs to be able to access /dev/pts/%i (i fought with trying to do this for a while, but not with devtmpfs)

Busybox blkid shows only LABEL and UUID fields, whereas "full" blkid shows also TYPE field. Will be shown that field after applying a patch?

yes, but Denys wanted to the submitter to use pointers, rather than duplicating code (it added unnecessary size) I understand, the whole pointer thing still gets me confused from time to time (well usually actually)

[goingnuts]

technosaurus:Excellent sum up of what to do and what not to do. Thanks!
I have used some days trying to merge squashfs-tools from v.3, v3.4 and v4.0 renaming almost every function, variable and constant in all of them - only to realize that the source code need to be split to make it work (multiple defined problem). But after giving up also realized that to cover pupngo with kernels from P216, P412, P430 and P525, you only need mksquashfs3.0, mksquashfs4.0 and unsquashfs4.0 (and I can merge mksquashfs3.0 and mksquashfs4.0). Reduced size from 917K to 455K (120K upxed).

Then almost made it with cdrkit-1.1.10 but got stuck in the below function from librols/snprintf.c where the missing function is format:

Code: Select all

#ifdef	PROTOTYPES
EXPORT int snprintf(char *buf, size_t maxcnt, const char *form, ...)
#else
EXPORT int snprintf(buf, maxcnt, form, va_alist)
	char	*buf;
	unsigned maxcnt;
	char	*form;
	va_dcl
#endif
{
	va_list	args;
	int	cnt;
	_BUF	bb;
	bb.ptr = buf;
	bb.count = maxcnt;
#ifdef	PROTOTYPES
	va_start(args, form);
#else
	va_start(args);
#endif
	cnt = format(_cput, (long)&bb, form,  args);
	va_end(args);
	if (maxcnt > 0)
		*(bb.ptr) = '\0';
	if (bb.count < 0)
		return (-1);
	return (cnt);
}

Anyone have an idea about which header to add or which library to link with?

[technosaurus]

snprintf is probably part of libc already, if so, you can probably comment it out.... or in this case maybe just add -DPROTOTYPES to CFLAGS

depending on the error (incompatible or multiple definition?)

[goingnuts]

The error is:

Code: Select all

 In function `snprintf':
snprintf.c:(.text.snprintf+0x28): undefined reference to `format'

Tried the -DPROTOTYPES but no change...
I can comment the line out, supply a snprinf.c from musl or totally exclude the snprintf from the objects being used - and finish the build...but don't know if its ok - the cdrkit-1.1.10 source tree is a little scattered and the comment in the header of the snprintf says:"This file has been modified for the cdrkit" suite.

Guess I just have to test the builded wodim with some burns.

Anyway - the multicallbins (mcb´s ?) are really great - have seen size reductions of 60% which almost equals to size of dynamic linked bins (without counting the size of the needed libs in).

[technosaurus]

Not all libc's have snprintf, so normally autotools builds this into either config.h or the makefile, so that it only builds on systems that don't have it. But as I am sure you have discovered, autotools suck... which is also why I used a standard shell build script (i would use makefile(s) if it grows much larger so that libs/objects get rebuilt iff changes occurred)

I haven't been working much on compiling and mcbs lately, b/c I am trying to write up some patents that involve using a device manager, so I have been studying mdev, udev, hotplug etc... and writing my own implementation in shell. This was helpful:
http://www.bangmoney.org/presentations/hotplug/
I expect that much of this will make its way into jwm_tools and bashbox for drive insertion etc... it turns out that it may even speed up the whole process because the previously mentioned tools run commands via sh -c <command> <args> but if I can do all of it in a shell using builtins to begin with it won't need to call another shell AND it won't use bash when it is linked to sh. I just wish I had more plugable devices to test.

[goingnuts]

...I am trying to write up some patents that involve using a device manager, so I have been studying mdev, udev, hotplug etc... and writing my own implementation in shell.[...]
I just wish I had more plugable devices to test.

Sounds good! If one could profile a plugable device (catch all generated files in proc/sys) and afterwards simulate insertion on another machine you could have access to plenty of plugable devices - but dont know if its possible.

[PANZERKOPF]

If one could profile a plugable device (catch all generated files in proc/sys) and afterwards simulate insertion on another machine you could have access to plenty of plugable devices - but dont know if its possible.

Catching can be easily performed but seems we cannot manually create or delete something in /sys directory to simulate nonexistent device.

[goingnuts]

Maybe a simulated input could just be read from a different directory (a fake /sys or /proc)?

Further on the mcb´s: Created attached mcb of Ace-of-Penguins1.3 - containing the games canfield, freecell, golf, mastermind, merlin, minesweeper, pegged, solitaire, spider, taipei and thornq (and additional the taipeditor taipedit). All the games uses libcard with is statically included. Upxed bin is 104K - 184K unpacked. Normal build would be 250-297K.
The mcb can be anywhere in your path and installed by command: "mcb_ace --install" which will create the needed symlinks (not needed for the attached pet though...).
Update 20190908: The original posted mcb of ace-1.3 is dynamic linked. The patch to give the mcb was lost and I recently recreated it and also updated relevant things from version 1.4. So attached my patch as well as a static linked version packed as pet. Should work on any Puppy version.

[goingnuts]

Another mcb - merging almost all bins from initrd.gz into one bin + some needed in main sfs for start up (udevd a.o.).
Contains:
udevd fuser puppyinputdetect guess_fstype hotplug2stdout hotplug2stdout_notimeout cddetect setserial fusermount elspci modprobe.bin modinfo ntfs-3g ntfs-3g.probe disktype resize2fs mke2fs mkfs.ext2 mkfs.ext3 fsck e2fsck fsck.ext2 fsck.ext3 find cp mount-FULL umount-FULL losetup rev
in 740K static build - thats more or less same size as dynamic linked bins exclusive needed libs. All programs are the full versions.

Place mcb_init in /bin and use command:"mcb_init --install" to create the symlinks. Symlinks are hardcoded to be created the usual places (/bin, /sbin, /usr/bin or /usr/sbin).

Its experimental and contains some dangerous disk tools - so use in non-critical environment. I did test most of the stuff but as always there is no guaranties.

[technosaurus]

Wow, you got ntfs3g too, I tried busybox-ing that one a year or so ago, and it almost seemed to be incompatible with itself. Excellent work.