pUPnGO - 6Mb ISO - Basic Building Block Puplet

[technosaurus]

thx, at least I know what is in store for me. I'd also like to make lib and include a bit more sane and clean up the need for various ../../../ garbage... using something like include/mi/mi.h and changing all references to be simply <mi/mi.h> or X11/mi/mi.h depending on where it _should_ be (what a PITA) - its hard to tell on some of them, many were split out from their parent directories into subdirectories to make things easier for someone... and a README in each dir is always useful - will put my findings, TODOs etc... there. My impression is that the xorg versions of kdrives were second class citizens toward the end, but if anyone can point me to a known sane version, I will try and pull from there too... I am kinda interested in their SDL version of kdrive (elephants and all)

[goingnuts]

@technosaurus: The ../../../ garbage... is my fault - not originally there. I think you could upgrade to 4.8 for the Xlib things and maybe leave the Xvesa part as is - or for a later round-up. I have added some functions to make gtk work - if needed I will upload my revised tool-chain? Ever thought of putting everything in one dir (source, headers, libs)?

[technosaurus]

Not necessarily 1 directory, but possibly 3 (include, libs, programs) I also think it would be a good idea to integrate my basic init with the xvesa,jwm,rxvt mcb because it would speed up booting. I figured out how to mount /dev/pts in c too so we are talking <1s boot times.

The only difference is I would just fork xvesa_main then call jwm_main as a function call without forking it and add this at the beginning in place of using an external mount binary.
#include <sys/mount.h>
mount( "/dev/pts", "/dev/pts", "devpts", MS_MGC_VAL, NULL );

This would allow the boot to X even without busybox ... Jwm needs command.c patched though if no shell is included (i have used fork...execvp)... It also cuts resource usage.

[goingnuts]

@technosaurus: Do you have a patch for the jwm exec-thing?

Did some test on the mcb_gtk to evaluate the influence on resources:
The bigger the mcb becomes the more ram is used at execution-time. Using upx does not give a huge difference (maybe +5-10% more ram is used).
So depending on the purpose one might use mcb (nothing beats that for size on disk), standalone (lowest ram usage
/largest size on disk) or the upxed variant of the two possibilities.
I have uploaded standalone version of sylpheed-1.0.6 and roxgtk1-2.2 as they for various reasons seems to be candidates for the standalone approach (sylpheed needs ssl and is big in itself/ROX is the only one using gdk-pixbuf). The ROX is patched to do pinboard-remove so now the puppypin-stuff is working. Also included most of the other needed files (icons/cddetect/diff) and a modified pup_event_frontend_d named roxpin.sh targeted this special version. Get them here

[Aitch]

techno/goingnuts

reading/watching you 2 develop is fascinating....I love the ideas and ways you are exploring - back to basics
I can't do coding at all, but it doesn't stop me from appreciating those who do
many thanks....look forward to a usable final minimalist version, with a brief instruction how to add[/remove?] software to taste

This maybe will suit my old IBM 560/570 that got put away with 214R on.....as being a bit slow......[I got an IBM T21, then T30/current]

thanks, guys

Aitch

[technosaurus]

I don't have the patch with me (still visiting family for the holidays) but basically it was replacing the sh -c at the end of src/command.c with

#define EGGSACK(a) ({if((fork())==0) execvp(a[0],a);})
EGGSACK(command);

Or simply (less secure?)
system(command);

BTW if we could setup a way to only keep one copy of the mcb running (the way rox does) then any newly executed apps could just be forked ... Which only needs to copy the rw data and would cut resource usage considerably... Adding it to my things to research.

[goingnuts]

Thanks. It would be a great step forward if that is possible. Do not know if its possible or practical but maybe a shared (but static) mcb library that a tiny launcher link to?

No doubt that this "issue" has to be solved in a way to minimize the resource use of heavier mcbs...

Made a mcb_gtk with the following content where size is reported for standalone builds (updated image as org contained errors in sum):

[nooby]

The following is not criticism and not a demand not even a friendly wish.

It is more like wanting to understand what is involved with the older
kernel. I don't ask you to do anything with pUPnGO but if somebody
else wants to do it. How difficult would it be? Is it like woof or much
more involved and very difficult?

I had promised myself to not bother you anymore.
I am too much of a total absolute Dummie so I know
I am annoying and much more but I love small distros.

Your distro is so cute.

I managed to get TinyCore Linux 4.1 going and it has drivers
that work for my hardware.

Can I borrow those drivers or the program that has them from
TCL and reuse them in your pUPnGO?

or is it easy to port your Puppy420 to say Lucid or something.
To change to another kernel.

No I don't ask you to do it at all. I mean is it easy for any of the other
developers to do it or would they have trouble doing it?

[technosaurus]

Any recommendations as to whether to use pipes, fifo, sockets or something else. I was thinking that when an mcb runs it could check to see if one of the above data streams exist and if not, then start it... Then this data stream could be used as a daemon to fork processes.

I've never really used them so...

Edit:
Did a bit of research and think named pipes (fifo) is the least complex and most versatilemethod. We could even just echo commands to the pipe once the daemon is running. This link had some useful info:
http://developers.sun.com/solaris/artic ... pipes.html
I have it figured out in shell, but need to translate it to C
It goes something like:
If ! -x /tmp/mcbfifo ... mkfifo /tmp/mcbfifo
While read line < /tmp/mcbfifo; do
Case $line in
...
Esac

[goingnuts]

@technosaurus: Would be cool and fast if manageable! dillo seems to do it with its dpis.
@nooby: What kernel do you prefer? Did you try the pUPnGO_V016_090611 with kernel from snowpup [2.6.35.7]?
No big deal to shift kernel in pupngo though - if you have a good reason giving you the energy and drive to do it...

[nooby]

Sadly I don't remember. I know I tested some of them.
But as I get it they all had the older kernel that lacked
the needed driver so most likely I did not test that one?

Because I use SnowPuppy 5 now and that one is based on
Lucid 513 and they use that kernel most likely. 2.6.35.7

I would have to find your kindly provided instructions then
to get it going again. Would be cool.

Just now I am too lazy but hope you can keep those
files somewhere so you don't delete them?

I would have to search through the thread and try to find where
you link to them unless that is on the first post?

Google found it for me
pUPnGO_V016_090611.iso si I have down loaded it .
Now I only need to find your descriptions on how to tell the CLI
what to do and find my way around.

Hehe New Years Eve is rescued from boredom of being totally alone.
Thanks for your answer

[technosaurus]

@nooby, my initramfs method works on all 2.6.X and 3.X kernels, and I have posted the tarballs, but few people are comfortable enough with the init process to basically rewrite it from scratch, so... I slowly chug along a few lines at a time.

@goingnuts
here is my basic init written in C, it pushes all of the more technical stuff into the X environment, in a way more typical of embedded systems

Code: Select all

//Copyright 2011 Brad Conroy.  Redistributable under the UIUC License.
#include <unistd.h>
#include <sys/wait.h>
#include <sys/mount.h>
#include <stdlib.h>

//these could be functions but gcc complains less this way 
#define EGGSACKWEIGHT(a) ({int s,p;if((p=fork())==0){execvp(a[0],a);}else{while(wait(&s)!= p);}})
#define SHIFTN(i,a) ({ int j=0;while(a[j] != NULL){a[j]=a[j+i];j++;}})
#define EGGSACK(a)	({if((fork())==0) execvp(a[0],a);})
int main(int argc, char** argv) {
char* X[99];

//Xvesa -screen 640x480x16 -nolisten tcp -tst -I &
	X[0]="Xvesa";
	X[1]="-screen";
	X[2]="800x600x16"; //low res for old boxes ... may need 640x480 - we'll see
	X[3]="-nolisten";
	X[4]="tcp";
	X[5]="-tst";	
	X[6]=NULL;
#ifdef MCB
	if((fork())==0) xvesa_main(6,X);
#else
	EGGSACK(X);
#endif

//Xvesa doesn't need any of this, so run it while X starts up
mount("/proc", "/proc", "proc", MS_MGC_VAL, NULL); //Ok if error, not in kernel.
mount("/sys", "/sys", "sys", MS_MGC_VAL, NULL); //Ok if error, not in kernel.
mount("/dev", "/dev", "devtmpfs", MS_MGC_VAL, NULL); //should fallback to tmpfs?
mount( "/dev/pts", "/dev/pts", "devpts", MS_MGC_VAL, NULL ); //for terminals

putenv("PATH=/bin");
putenv("HOME=/");
putenv("TERM=rxvt");
putenv("PS1=# ");
putenv("APP_DIR=/usr/share/ROX-Filer");
putenv("CHOICESPATH=/usr/share");
putenv("SHELL=/bin/sh");
putenv("DISPLAY=:0");

X[0]="ROX-Filer";
X[1]="-p";
X[2]="default";
X[3]=NULL;
#ifdef MCB
int s,p; //MCB block not yet tested
if((p=fork())==0) rox_main(3,X);
#else
EGGSACK(X);
#endif

X[0]="jwm";
X[1]="-display";
X[2]=":0";
X[3]=NULL;
#ifdef MCB
int s,p; //MCB block not yet tested
if((p=fork())==0) {jwm_main(3,X);}else{{while(wait(&s)!= p);}
#else
EGGSACKWEIGHT(X);
#endif
//kill Xvesa here? or have to Ctrl+Alt+Backspace

X[0]="sh";
X[1]=NULL;
execvp(X[0],X); //this is the last process, so let /bin/sh be init now
}

I tried working on a pipe for the MCBs, but kinda got stuck:

Code: Select all

#include <stdio.h>
#include <errno.h>
#include <ctype.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define MCB_PIPE	"/tmp/.mcbpipe"
#define MAX_BUF_SIZE	255
#define EGGSACK(a) ({if((fork())==0) execvp(a[0],a);}) 

void read_pipe(int fd){
	int numread=0,i=0,j=0,k=0;
    char buf[MAX_BUF_SIZE], **myargs;
    while (1) {
		i=0;j=0;k=0;
		numread = read(fd, buf, MAX_BUF_SIZE);
		buf[numread] = '\0';
		if (numread > 0 ){
			//for debugging
			printf("%d,%c,%s",numread, buf[1], buf);
		}
		sleep(1);
    }	
}

int main(int argc, char *argv[])
{
    int fd, ret_val;
    char buf[MAX_BUF_SIZE];

    ret_val = mkfifo(MCB_PIPE, 0666);    //Create pipe
    if ((ret_val == -1) && (errno != EEXIST)) {
        perror("Error creating the named pipe");
        return (1);
    }

//if (errno == EEXIST) return(printf("write argv to pipe\n"));
//else start daemon reading from pipe to run as commands

    fd = open(MCB_PIPE, O_RDONLY);    //Open pipe to read
    read_pipe(fd);    //Read pipe
}

I can
echo some stuff >/tmp/.mcbpipe
and the debugging printf will print some stuff,... but what to do from there?
special formatting, so it can be converted to argv format maybe?

[goingnuts]

technosaurus: Nice and lean init-framework!
I too blew in the pipes but got nowhere - I can trigger a call to the mcb_gtk and fork an application but that does not reduce memory footprint - its still just the hole mcb being loaded...
Now the memory usage reported earlier might not be true - there might be some sharing between the apps - but still you have to load everything when the first app is to run. Some busybox test reported here..BB does it too.

Tried to convert the mcb to a shared lib - the idea was to have a shared lib having everything the apps needed and then have a tiny launcher-app pulling what was needed from the lib. But no success - dont know if it possible at all?

The --gc-sections seems effective to pull out most of the unneeded from the mcb but how to do the upper side? Create a library only holding whats needed for the apps in the mcb?

To prevent freezing one can set this at boot:
echo -n 100 > /proc/sys/vm/overcommit_ratio
echo -n 1 > /proc/sys/vm/oom_kill_allocating_task

works ok but might give unexpected results ref.
Even tried to do it with a shell script - with the plan to kill latest pid always if free memory got too low... ..not one of my greatest ideas.

[technosaurus]

Tried to convert the mcb to a shared lib - the idea was to have a shared lib having everything the apps needed and then have a tiny launcher-app pulling what was needed from the lib. But no success - dont know if it possible at all?

The --gc-sections seems effective to pull out most of the unneeded from the mcb but how to do the upper side? Create a library only holding whats needed for the apps in the mcb?

if you want to know what sections get removed (so you can trim the source code) try --print-gc-sections (with --gc-sections)
it will tell you about all of the dead code that can be removed
once they are physically removed, there is no need for --gc-sections and a library can be built using -shared -Wl,-soname,libmcb.so.0 -o libmcb.so.0.0 then symlinking to *.so and *.so.0 (just use the *_main functions and their dependencies) This will allow you to do an mcb or a main() for each app ... ex:

Code: Select all

int main(int argc, char **argv){
return(jwm_main(argc,argv))
}

as for the _actual_ memory used; I posted meminfo earlier, which I needed because I compiled my kernel without /proc or /sys
running Xvesa at 800x600x16 with jwm, rox, rxvt, dillo, xdialog, mp and mtpaint only shows <12mb ram usage just using the symlinks
Anyhow it seems to do a better job reporting actual system ram used ... though not for individual processes ... shared "pages" or something
(btw I designed its output to be easy to use in scripts with eval `meminfo`)

EDIT: the gtk1 version of gtk-server would be a really great addition to the gtk1 mcb (though I bet using --gc-sections will break it) - there are quite a few games/utilities already and many of us are more comfortable writing shell scripts than compiling C (myself included, though I am getting better). Also IIRC you made some improvements to the ashirc script that I grokked, that would be nice to have, but I was think of adding some kind of bot-type thing to it for testing (output lsmod, lspci, lsusb ...) and bug reporting... then I could keep a log of the channel to scrub for data (what works, what doesn't)

PS - I am rewriting bashbox (not completely, just adding things in 1 by 1 and auditing the code as I do) for these recent improvements and will add a launch() function that can do these memory related tasks (amongst others)... but its looking like I need to implement a resource usage element for packages ... otherwise launch() would need to track it on firstrun of each app (but when it uses under 12mb with all apps running, it may not be a big concern right now)

[amigo]

Hello zealots! I've been lurking around here the last couple of days a bit. Can you clue me in on this 'mcb' thing? Please don't make me read all 36 pages of the thread...

Technosaurus, you asked a while back about why init was not killable. This is because the kernel hands init the first process number -which is supposed to be a daemon which never dies and keeps everything else running which is supposed to start and keep-started everything else which is needed. Executing programs directly as part of the internal init code is probably not a good idea in the long run -very inflexible as well.

I've downloaded the pet of tiny-x11 sources -when I have a chance I'll look at it and see if there are fixes/changes which can be 'upstreamed' to my hacked sources.

[goingnuts]

@technosaurus: I tried to make the gtk-server but unfortunately I only can get it running with gtk2 - gets the following error with gtk1:
./demo-stdin.awk
*** glibc detected *** gtk-server: free(): invalid pointer: 0xb7ea0f04 ***

@amigo: Maybe technosaurus can do a better explanation but in short it just a multicall binary bundling several applications in one bin. On this page top you can see that the 9 apps compiled as standalone bins (static) weighs around 11Mb and in a MBC they get down to 4Mb (upxed 1,5Mb). So the MCB contains everything needed for the 9 apps to run (libc/libX/glib/gdk/gtk etc) but not configurations files or external images if they need that.
The single application is called via a symlink to the MCB-bin which spawns the right application - just like busybox.
Most size reduction is obtained with applications needing the same stuff and build static as far as I can see...

[amigo]

Yeah, as I was doing something else this afternoon I realized that the reference was to multi-call binary. I was wondering where the sources are??

As to gtk-server for gtk1, you have to use the older version of the sources. If you can't find them elsewhere, I have them here:
http://distro.ibiblio.org/amigolinux/do ... ver-1.3.4/

[goingnuts]

Yeah, as I was doing something else this afternoon I realized that the reference was to multi-call binary. I was wondering where the sources are??

As to gtk-server for gtk1, you have to use the older version of the sources. If you can't find them elsewhere, I have them here:
http://distro.ibiblio.org/amigolinux/do ... ver-1.3.4/

I can upload some sources on request for the things I have made. technosaurous has published a more generic source some pages back. Also attached a small demo to build minimp3 and wavplay (standalone or as a mcb).

Thanks for the link to the gtk-server - but it gives same problem. Tried your binary as well with same result. Might be a glib problem in this end but I have tired to recompile and reinstall so sort of stuck.
Some warnings during gtk-server compile:
"from funcs.c:3:
/usr/local/include/glib-1.2/glib.h23: warning: ISO C does not permit named variadic macros"

Maybe I should use an older compiler?

[amigo]

Yeah, an older compiler would probably do the trick -I'd try 4.1.2 or 3.4.6 first, but I seem to remember that error from old written-for-gcc-2.95-3 sources.... yukk!

Thanks for the link to the proper page of this thread -that's what I was looking for.

[technosaurus]

I have had pretty good luck with 3.4.6, it even builds against dietlibc(not that I would use diet). Anything after gcc 4.2 has extra build requirements not to mention gpl3.

I pulled out glade1 and glade2 to see how easy it would be, would anyone like a tutorial(and pets), so they can make compile-able GUIs?