pUPnGO - 6Mb ISO - Basic Building Block Puplet

[technosaurus]

I don't have the patch with me (still visiting family for the holidays) but basically it was replacing the sh -c at the end of src/command.c with

#define EGGSACK(a) ({if((fork())==0) execvp(a[0],a);})
EGGSACK(command);

Or simply (less secure?)
system(command);

BTW if we could setup a way to only keep one copy of the mcb running (the way rox does) then any newly executed apps could just be forked ... Which only needs to copy the rw data and would cut resource usage considerably... Adding it to my things to research.

[goingnuts]

Thanks. It would be a great step forward if that is possible. Do not know if its possible or practical but maybe a shared (but static) mcb library that a tiny launcher link to?

No doubt that this "issue" has to be solved in a way to minimize the resource use of heavier mcbs...

Made a mcb_gtk with the following content where size is reported for standalone builds (updated image as org contained errors in sum):

[nooby]

The following is not criticism and not a demand not even a friendly wish.

It is more like wanting to understand what is involved with the older
kernel. I don't ask you to do anything with pUPnGO but if somebody
else wants to do it. How difficult would it be? Is it like woof or much
more involved and very difficult?

I had promised myself to not bother you anymore.
I am too much of a total absolute Dummie so I know
I am annoying and much more but I love small distros.

Your distro is so cute.

I managed to get TinyCore Linux 4.1 going and it has drivers
that work for my hardware.

Can I borrow those drivers or the program that has them from
TCL and reuse them in your pUPnGO?

or is it easy to port your Puppy420 to say Lucid or something.
To change to another kernel.

No I don't ask you to do it at all. I mean is it easy for any of the other
developers to do it or would they have trouble doing it?

[technosaurus]

Any recommendations as to whether to use pipes, fifo, sockets or something else. I was thinking that when an mcb runs it could check to see if one of the above data streams exist and if not, then start it... Then this data stream could be used as a daemon to fork processes.

I've never really used them so...

Edit:
Did a bit of research and think named pipes (fifo) is the least complex and most versatilemethod. We could even just echo commands to the pipe once the daemon is running. This link had some useful info:
http://developers.sun.com/solaris/artic ... pipes.html
I have it figured out in shell, but need to translate it to C
It goes something like:
If ! -x /tmp/mcbfifo ... mkfifo /tmp/mcbfifo
While read line < /tmp/mcbfifo; do
Case $line in
...
Esac

[goingnuts]

@technosaurus: Would be cool and fast if manageable! dillo seems to do it with its dpis.
@nooby: What kernel do you prefer? Did you try the pUPnGO_V016_090611 with kernel from snowpup [2.6.35.7]?
No big deal to shift kernel in pupngo though - if you have a good reason giving you the energy and drive to do it...

[nooby]

Sadly I don't remember. I know I tested some of them.
But as I get it they all had the older kernel that lacked
the needed driver so most likely I did not test that one?

Because I use SnowPuppy 5 now and that one is based on
Lucid 513 and they use that kernel most likely. 2.6.35.7

I would have to find your kindly provided instructions then
to get it going again. Would be cool.

Just now I am too lazy but hope you can keep those
files somewhere so you don't delete them?

I would have to search through the thread and try to find where
you link to them unless that is on the first post?

Google found it for me
pUPnGO_V016_090611.iso si I have down loaded it .
Now I only need to find your descriptions on how to tell the CLI
what to do and find my way around.

Hehe New Years Eve is rescued from boredom of being totally alone.
Thanks for your answer

[technosaurus]

@nooby, my initramfs method works on all 2.6.X and 3.X kernels, and I have posted the tarballs, but few people are comfortable enough with the init process to basically rewrite it from scratch, so... I slowly chug along a few lines at a time.

@goingnuts
here is my basic init written in C, it pushes all of the more technical stuff into the X environment, in a way more typical of embedded systems

Code: Select all

//Copyright 2011 Brad Conroy.  Redistributable under the UIUC License.
#include <unistd.h>
#include <sys/wait.h>
#include <sys/mount.h>
#include <stdlib.h>

//these could be functions but gcc complains less this way 
#define EGGSACKWEIGHT(a) ({int s,p;if((p=fork())==0){execvp(a[0],a);}else{while(wait(&s)!= p);}})
#define SHIFTN(i,a) ({ int j=0;while(a[j] != NULL){a[j]=a[j+i];j++;}})
#define EGGSACK(a)	({if((fork())==0) execvp(a[0],a);})
int main(int argc, char** argv) {
char* X[99];

//Xvesa -screen 640x480x16 -nolisten tcp -tst -I &
	X[0]="Xvesa";
	X[1]="-screen";
	X[2]="800x600x16"; //low res for old boxes ... may need 640x480 - we'll see
	X[3]="-nolisten";
	X[4]="tcp";
	X[5]="-tst";	
	X[6]=NULL;
#ifdef MCB
	if((fork())==0) xvesa_main(6,X);
#else
	EGGSACK(X);
#endif

//Xvesa doesn't need any of this, so run it while X starts up
mount("/proc", "/proc", "proc", MS_MGC_VAL, NULL); //Ok if error, not in kernel.
mount("/sys", "/sys", "sys", MS_MGC_VAL, NULL); //Ok if error, not in kernel.
mount("/dev", "/dev", "devtmpfs", MS_MGC_VAL, NULL); //should fallback to tmpfs?
mount( "/dev/pts", "/dev/pts", "devpts", MS_MGC_VAL, NULL ); //for terminals

putenv("PATH=/bin");
putenv("HOME=/");
putenv("TERM=rxvt");
putenv("PS1=# ");
putenv("APP_DIR=/usr/share/ROX-Filer");
putenv("CHOICESPATH=/usr/share");
putenv("SHELL=/bin/sh");
putenv("DISPLAY=:0");

X[0]="ROX-Filer";
X[1]="-p";
X[2]="default";
X[3]=NULL;
#ifdef MCB
int s,p; //MCB block not yet tested
if((p=fork())==0) rox_main(3,X);
#else
EGGSACK(X);
#endif

X[0]="jwm";
X[1]="-display";
X[2]=":0";
X[3]=NULL;
#ifdef MCB
int s,p; //MCB block not yet tested
if((p=fork())==0) {jwm_main(3,X);}else{{while(wait(&s)!= p);}
#else
EGGSACKWEIGHT(X);
#endif
//kill Xvesa here? or have to Ctrl+Alt+Backspace

X[0]="sh";
X[1]=NULL;
execvp(X[0],X); //this is the last process, so let /bin/sh be init now
}

I tried working on a pipe for the MCBs, but kinda got stuck:

Code: Select all

#include <stdio.h>
#include <errno.h>
#include <ctype.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define MCB_PIPE	"/tmp/.mcbpipe"
#define MAX_BUF_SIZE	255
#define EGGSACK(a) ({if((fork())==0) execvp(a[0],a);}) 

void read_pipe(int fd){
	int numread=0,i=0,j=0,k=0;
    char buf[MAX_BUF_SIZE], **myargs;
    while (1) {
		i=0;j=0;k=0;
		numread = read(fd, buf, MAX_BUF_SIZE);
		buf[numread] = '\0';
		if (numread > 0 ){
			//for debugging
			printf("%d,%c,%s",numread, buf[1], buf);
		}
		sleep(1);
    }	
}

int main(int argc, char *argv[])
{
    int fd, ret_val;
    char buf[MAX_BUF_SIZE];

    ret_val = mkfifo(MCB_PIPE, 0666);    //Create pipe
    if ((ret_val == -1) && (errno != EEXIST)) {
        perror("Error creating the named pipe");
        return (1);
    }

//if (errno == EEXIST) return(printf("write argv to pipe\n"));
//else start daemon reading from pipe to run as commands

    fd = open(MCB_PIPE, O_RDONLY);    //Open pipe to read
    read_pipe(fd);    //Read pipe
}

I can
echo some stuff >/tmp/.mcbpipe
and the debugging printf will print some stuff,... but what to do from there?
special formatting, so it can be converted to argv format maybe?

[goingnuts]

technosaurus: Nice and lean init-framework!
I too blew in the pipes but got nowhere - I can trigger a call to the mcb_gtk and fork an application but that does not reduce memory footprint - its still just the hole mcb being loaded...
Now the memory usage reported earlier might not be true - there might be some sharing between the apps - but still you have to load everything when the first app is to run. Some busybox test reported here..BB does it too.

Tried to convert the mcb to a shared lib - the idea was to have a shared lib having everything the apps needed and then have a tiny launcher-app pulling what was needed from the lib. But no success - dont know if it possible at all?

The --gc-sections seems effective to pull out most of the unneeded from the mcb but how to do the upper side? Create a library only holding whats needed for the apps in the mcb?

To prevent freezing one can set this at boot:
echo -n 100 > /proc/sys/vm/overcommit_ratio
echo -n 1 > /proc/sys/vm/oom_kill_allocating_task

works ok but might give unexpected results ref.
Even tried to do it with a shell script - with the plan to kill latest pid always if free memory got too low... ..not one of my greatest ideas.

[technosaurus]

Tried to convert the mcb to a shared lib - the idea was to have a shared lib having everything the apps needed and then have a tiny launcher-app pulling what was needed from the lib. But no success - dont know if it possible at all?

The --gc-sections seems effective to pull out most of the unneeded from the mcb but how to do the upper side? Create a library only holding whats needed for the apps in the mcb?

if you want to know what sections get removed (so you can trim the source code) try --print-gc-sections (with --gc-sections)
it will tell you about all of the dead code that can be removed
once they are physically removed, there is no need for --gc-sections and a library can be built using -shared -Wl,-soname,libmcb.so.0 -o libmcb.so.0.0 then symlinking to *.so and *.so.0 (just use the *_main functions and their dependencies) This will allow you to do an mcb or a main() for each app ... ex:

Code: Select all

int main(int argc, char **argv){
return(jwm_main(argc,argv))
}

as for the _actual_ memory used; I posted meminfo earlier, which I needed because I compiled my kernel without /proc or /sys
running Xvesa at 800x600x16 with jwm, rox, rxvt, dillo, xdialog, mp and mtpaint only shows <12mb ram usage just using the symlinks
Anyhow it seems to do a better job reporting actual system ram used ... though not for individual processes ... shared "pages" or something
(btw I designed its output to be easy to use in scripts with eval `meminfo`)

EDIT: the gtk1 version of gtk-server would be a really great addition to the gtk1 mcb (though I bet using --gc-sections will break it) - there are quite a few games/utilities already and many of us are more comfortable writing shell scripts than compiling C (myself included, though I am getting better). Also IIRC you made some improvements to the ashirc script that I grokked, that would be nice to have, but I was think of adding some kind of bot-type thing to it for testing (output lsmod, lspci, lsusb ...) and bug reporting... then I could keep a log of the channel to scrub for data (what works, what doesn't)

PS - I am rewriting bashbox (not completely, just adding things in 1 by 1 and auditing the code as I do) for these recent improvements and will add a launch() function that can do these memory related tasks (amongst others)... but its looking like I need to implement a resource usage element for packages ... otherwise launch() would need to track it on firstrun of each app (but when it uses under 12mb with all apps running, it may not be a big concern right now)

[amigo]

Hello zealots! I've been lurking around here the last couple of days a bit. Can you clue me in on this 'mcb' thing? Please don't make me read all 36 pages of the thread...

Technosaurus, you asked a while back about why init was not killable. This is because the kernel hands init the first process number -which is supposed to be a daemon which never dies and keeps everything else running which is supposed to start and keep-started everything else which is needed. Executing programs directly as part of the internal init code is probably not a good idea in the long run -very inflexible as well.

I've downloaded the pet of tiny-x11 sources -when I have a chance I'll look at it and see if there are fixes/changes which can be 'upstreamed' to my hacked sources.

[goingnuts]

@technosaurus: I tried to make the gtk-server but unfortunately I only can get it running with gtk2 - gets the following error with gtk1:
./demo-stdin.awk
*** glibc detected *** gtk-server: free(): invalid pointer: 0xb7ea0f04 ***

@amigo: Maybe technosaurus can do a better explanation but in short it just a multicall binary bundling several applications in one bin. On this page top you can see that the 9 apps compiled as standalone bins (static) weighs around 11Mb and in a MBC they get down to 4Mb (upxed 1,5Mb). So the MCB contains everything needed for the 9 apps to run (libc/libX/glib/gdk/gtk etc) but not configurations files or external images if they need that.
The single application is called via a symlink to the MCB-bin which spawns the right application - just like busybox.
Most size reduction is obtained with applications needing the same stuff and build static as far as I can see...

[amigo]

Yeah, as I was doing something else this afternoon I realized that the reference was to multi-call binary. I was wondering where the sources are??

As to gtk-server for gtk1, you have to use the older version of the sources. If you can't find them elsewhere, I have them here:
http://distro.ibiblio.org/amigolinux/do ... ver-1.3.4/

[goingnuts]

Yeah, as I was doing something else this afternoon I realized that the reference was to multi-call binary. I was wondering where the sources are??

As to gtk-server for gtk1, you have to use the older version of the sources. If you can't find them elsewhere, I have them here:
http://distro.ibiblio.org/amigolinux/do ... ver-1.3.4/

I can upload some sources on request for the things I have made. technosaurous has published a more generic source some pages back. Also attached a small demo to build minimp3 and wavplay (standalone or as a mcb).

Thanks for the link to the gtk-server - but it gives same problem. Tried your binary as well with same result. Might be a glib problem in this end but I have tired to recompile and reinstall so sort of stuck.
Some warnings during gtk-server compile:
"from funcs.c:3:
/usr/local/include/glib-1.2/glib.h23: warning: ISO C does not permit named variadic macros"

Maybe I should use an older compiler?

[amigo]

Yeah, an older compiler would probably do the trick -I'd try 4.1.2 or 3.4.6 first, but I seem to remember that error from old written-for-gcc-2.95-3 sources.... yukk!

Thanks for the link to the proper page of this thread -that's what I was looking for.

[technosaurus]

I have had pretty good luck with 3.4.6, it even builds against dietlibc(not that I would use diet). Anything after gcc 4.2 has extra build requirements not to mention gpl3.

I pulled out glade1 and glade2 to see how easy it would be, would anyone like a tutorial(and pets), so they can make compile-able GUIs?

[Aitch]

Techno

Never say no to a tutorial, even if I don't always understand it for ages...

that's a 'yes' by the way

Aitch

[PANZERKOPF]

I tried to make the gtk-server but unfortunately I only can get it running with gtk2 - gets the following error with gtk1:
./demo-stdin.awk
*** glibc detected *** gtk-server: free(): invalid pointer: 0xb7ea0f04 ***

Weird..
I have compiled old version of gtk-server(1.3.4) with gtk1 and GCC 4.4.4.
Successfully tested with some demo-scripts included in source package.

[technosaurus]

Same here, but I used the Debian patches for gtk1 iirc, I need to archive those patches before they purge them (the patches fixed a lot of compile issues)

[goingnuts]

Thanks - its nice to know that it can be done - although still no luck: tried gcc 4.2.2 (std. in P412), gcc 3.3 and gcc 2.85, rebuild glib, gtk and gtk server but still same error - also tried the debian-old source... But as long as it can be done there is some hope - think I will leave it for now though.

@technosaurus: You could modify your init with the below to have an init running:

Code: Select all

//X[0]="ash";
//X[1]=NULL;
//execvp(X[0],X);
pid_t pid;
pid = fork();
if (pid == 0) {X[0]="/bin/ash"; X[1]=NULL; execvp(X[0],X);}
while (1) {sleep(1);}
return 0;

[Aitch]

Guys
Have you seen Iguleder's homepage?
http://www.dimakrasner.com/
I'd be interested in your views on Arm port/xtoolchain/building of puppy
Seems to me if techno, goingnuts, amigo, Iguleder, and big_bass got together a 'wee beasty' would emerge that might run on x86/86/64/arm/ppc etc and was 'built properly....' and could be maintained/updated easily....or am I dreaming again?

Aitch