Woof-based Puppy builders wanted

[ttuuxxx]

Just to be clear.. I am not gone.

I have ideas but I severely lack time and that will probably be the case for a couple to several more months.

Just to capture BK's vision of woof..

For a long time I have dreamt of a "magical script" that could download packages of some other distro, cut them right down to Puppy-size, then build a Puppy Linux live-CD -- and do all of this totally automatically.

(Ref)

-----------------------------------

While I am scarce I am contactable, though response may not be immediate. Feel free to contact me, especially future distro builders.

Cheers.

I'm not gone either, If I'm needed I will make the time to help out. I would rather help out and give Idea's.
Jeff

[mavrothal]

I'm not gone either, If I'm needed I will make the time to help out. I would rather help out and give Idea's.
Jeff

Nice to see you're 'round ttuuxxx
Help and ideas is always good, though puppy builders (ie release managers/lead developers) is what puppy needs most right now I think.
Few years back you have expressed some interest for puppy 4.2/5 and then latter puppy 6. What about puppy 7?...

[ttuuxxx]

I'm not gone either, If I'm needed I will make the time to help out. I would rather help out and give Idea's.
Jeff

Nice to see you're 'round ttuuxxx
Help and ideas is always good, though puppy builders (ie release managers/lead developers) is what puppy needs most right now I think.
Few years back you have expressed some interest for puppy 4.2/5 and then latter puppy 6. What about puppy 7?...

I've compiled just about every aspect of puppy in the past, But haven't had much free time on my hands ever since I started my last job 3yrs ago and had a daughter and went back to school at nights and my wife does 24hr shifts, lol I've been very busy. But I could work with a developer to share the load and get things off the ground. I'm a bit rusty, haven't really done much linux do to the time thing, but I'm finished school in August, which will free up some time, If there is a real need, I'll be here to help.
Did Barry really retire, he talked about it for years, but I haven't see the notice.
Jeff

[mavrothal]

[
Did Barry really retire, he talked about it for years, but I haven't see the notice.

Do not know about retirement but certainly looks much less active.
First he put woof in maintenance mode, then he suggested woof-CE for puppy development, then he started working on a new quirky based in T2 and Ubuntu Tahr alpha, finally a month ago put out his quirky build system and since then he is quite.
Who knows what will happen next, but I would keep my hopes low.

[stemsee]

It seems to me that DebianDog is the place where it is all happening! With tireless maintaners, coordination, energy, innovation and talent.

"When one door closes..."

[starhawk]

I wish I could help. I've got a couple ideas for some interesting (to me) Pups... but I barely know enough to troubleshoot things that I accidentally break. Knowledge of how things work is one of the big things, if not the biggest, that separates user from dev -- and by that measure, I am most definitely a user at this time...

I guess what I'm asking is, how and where can I learn the highly technical details of how Puppy functions...? I can read bash script (with a little help from Google) and I've plenty of time...

[jamesbond]

I guess what I'm asking is, how and where can I learn the highly technical details of how Puppy functions...? I can read bash script (with a little help from Google) and I've plenty of time...

Really? Ok, here we go.

---

Step 1: Read this: https://github.com/iguleder/roar-ng-ii/ ... ramfs/init. This is roar-ng-ii's /init script (Roar NG II was the minimalist distro Iguleder worked before he switched his attention to LazyUx and packdude - I wanted to show you the /init script from Lazyux but I can't find any).

It is the first program executed by a waking-up kernel (everything before that is "boot loader" and "kernel initialisation" which is from the realm beyond puppy).

This script, at 287 lines, is a lot shorter and simpler than Puppy's init script but it does more or less the same thing, if you can understand this then it would help you to better grasp puppy's /init script. It is also beautifully written and is heavily commented to help with your understanding.

---

Step 2: The last line of /init will execute /sbin/init, which is a busybox binary. /sbin/init will do actions according to what you tell it to do in /etc/inittab, so go and read https://github.com/iguleder/roar-ng-ii/ ... tc/inittab. Notice the first few lines.

---

Step 3: Ahh, so /etc/inittab instructs /sbin/init to execute /etc/rc.d/rc.sysinit as the first thing to do! May as well read what what rc.sysinit does ... https://github.com/iguleder/roar-ng-ii/ ... rc.sysinit.

---

Step 4: Finished already? Don't get the clue on what's next? Well as soon as rc.sysinit finishes, it returns back to /sbin/init, which will find stuff to do. Hint: look at those tty1, tty2 lines ... hmm, all of them seems to be running getty (="get tty", which runs login, which display the user/name prompt); but Puppy never ask me for username and password. Perhaps we should look at Puppy's *real* inittab https://github.com/puppylinux-woof-CE/w ... tc/inittab ...

Eh, what's this? Puppy's inittab for tty1 says "mingetty --autologin". That's not the same as roar-ng-ii's. Mingetty is a replacement for getty, and "--autologin" means what exactly what it is: "login automatically". So what happens after one logins? the "login" program will launch the "login shell".

"login shell" is just a normal "shell" (ie, /bin/sh) that will do certain things because the shell executed at "login time". There are other deeper meanings, but let's skimp on them at the moment.

A standard, posix "login shell" will first execute ("sources", to be exact) anything in /etc/profile, so lets look at that ... https://github.com/iguleder/roar-ng-ii/ ... tc/profile.


---

Step 5: When done, the "login shell" will also run $HOME/.profile (in this case, user is "root" so its $HOME is "/root", so lets look at roar-ng-ii's root folder ... https://github.com/iguleder/roar-ng-ii/ ... ootfs/root.

Eh! Roar-ng-ii's root folder is empty, there is no .profile !!! So what happens next?

What happens next, is, the shell becomes interactive (=ie accepts input, waiting for your command) and display the familiar "#" prompt that you love and see from terminal. You've dropped at that Linux console. This is something that you only see if you choose to "Exit X" from jwm, usually accompanied by scary messages about how to run xorgwizard if you can't get the graphical desktop started again - but be assured, there is nothing scary here.

Hang one a minute, you say. Puppy does *NOT* stop at Linux console, not unless *I* tell it to stop by giving pfix=nox as the boot parameter. You can't be right! Well, my dear, you are looking at roar-ng-ii's /root/.profile here, and indeed if you boot roar-ng-ii you will be dropped at the linux console. Perhaps then we should look at *Puppy*'s /root/.profile, here: https://github.com/puppylinux-woof-CE/w ... t/.profile, should we?

Aha !!! It is *definitely* not empty. And what it is near the end of the file? ... it runs "xwin" ... That's it, it starts the graphical desktop !

---

Step 6: What's next? There is still so much story to tell, but before that, we need to step back and re-trace our path. In most of the steps above, we have been examining roar-ng-ii's files. The order is identical, the commands that get executed is identical, the rough purpose of the scripts are identical - I hope by this stage you have a clear understanding of what every script does.

When you are ready, you should then feel brave enough to look at the real thing:
Puppy's init https://github.com/puppylinux-woof-CE/w ... tree0/init - from Step 1
Puppy's inittab https://github.com/iguleder/roar-ng-ii/ ... tc/inittab from step 2.
Puppy's rc.sysinit https://github.com/puppylinux-woof-CE/w ... rc.sysinit from step 3
Puppy's /etc/profile https://github.com/puppylinux-woof-CE/w ... tc/profile from step 4
And you you have already reviewed Puppy's /root/.profile from step 5.

And when you feel you have understood them, you will see scripts from above will call other scripts - make sure you go and read them and understand them too. Only when you've done that, you will be prepared to continue the journey to the wonders of Graphical Desktop and Puppy event system.

---

To be continued (when I'm motivated enough and have enough free time).

EDIT: typos.

[jamesbond]

Just to be clear.. I am not gone.

Not sure if this is a good thing or a bad thing
Looks like you scared everyone away

[starhawk]

...that is a lot of reading.

Let me grab some lunch and I'll get started... can't concentrate on an empty stomach!

[stemsee]

@JamesBond

That's just what I was looking for ... but didn't know how to ask?

[starhawk]

@jamesbond -- started reading that first script, and my brain melted a little.

I don't think today's a day I should be trying to parse that stuff. I've bookmarked it for later...

[jamesbond]

Mavrothal sparked the interested by posting this thread. Let me add to it: how about building puppy from tahr/utopic, *directly from ubuntu repo*, no conversion to PET required?

I have a proof-of-concept under-300-lines script that builds puppy.sfs *directly* from Ubuntu, tested with trusty and utopic (32-bit).
It uses genuine Woof-CE puppy scripts and genuine Woof-CE initrd (kernel is Fatdog's but that's because I'm lazy). So far it builds command-line only (bash, dash, tar, grep, gawk), but it's easy to extend. Boots under qemu.

Note:
1. This is proof-of-concept and *unfinished work*.
2. Only builds puppy.sfs. The initrd I used for testing is manually prepared (easy, only 3 steps - copy DISTRO_SPEC, edit DISTRO_SPEC, comment out one stupid line in /init, then usual cpio)
3. Tested with trusty and utopic, it should be easy to extend this to Debian too (any versions).
4. The only requirement is you need to have recent-enough full version of "dpkg" (and dpkg-deb) installed (easy, if you're under ubuntu-based puppy just "apt-get dpkg"; for other puppies just compile from source)

5. Script *only builds* from Debian-style packages. Don't ask for building from slackware packages, RPM packages, etc - question like that > /dev/null.
6. You still need Woof-CE as all the puppy core lives there.
7. Uses native Debian package manager. Does not use PET.

Any interest? Comments?

[stemsee]

Built entirely from .debs no pets?
Your 64 bit kernel?
So resulting sfs has full xorg/xinput etc?

[neerajkolte]

*directly from ubuntu repo*, no conversion to PET required?
Uses native Debian package manager. Does not use PET.

Seems very interesting.
For me having and maintaining pet packages repos seemed like having a parallel universe.
If it uses native debian package manager and take care of dependencies that would really be great.

Then again dosen't Debiandog do exactly that.
I haven't used it but it says so.

[jamesbond]

Built entirely from .debs no pets?

Yes.

Your 64 bit kernel?

As a stop-gap. We should use proper 32-bit kernel instead.

So resulting sfs has full xorg/xinput etc?

I have not included Xorg in the build yet, but when I do, then plan is to bring use Ubuntu's native Xorg. Thus, if Ubuntu's xorg support xinput, then this will do, too.

If it uses native debian package manager and take care of dependencies that would really be great.

Yes, it does. It uses native debian package manager (dpkg) and it does take care of dependencies. Dpkg is not the one that usually sees though, people usually use apt-get or synaptics (both are one-step higher than dpkg and uses dpkg internally). I plan to get there too.

Then again dosen't Debiandog do exactly that.
I haven't used it but it says so.

Hahaha, yeah. Many roads to Rome, eh? Debiandog takes an existing Debian LiveCD and converts it to puppy. The result is fully Debian to the core, yet feels and works like puppy. Saintless has done a good work on that.
Me, I take the Ubuntu (or Debian) packages and build them up. The result is Puppy to the core, yet will work and is fully compatible with Ubuntu/Debian (I hope )

[mavrothal]

I have a proof-of-concept under-300-lines script that builds puppy.sfs *directly* from Ubuntu, tested with trusty and utopic (32-bit).
It uses genuine Woof-CE puppy scripts and genuine Woof-CE initrd

<snip>

6. You still need Woof-CE as all the puppy core lives there.
7. Uses native Debian package manager. Does not use PET.

Any interest? Comments?

Yeh I'm interested
Puppy does use a number of pets though not in woof and characteristic to puppy (jwmconfig for example).
However this could be fixed bu adding them flat into the recently added woof-CE/woof-code/rootfs-packages directory
Does you 300-liner considers rootfs-packages and if it does does it also runs any pinstall included there?

This can be a really interesting development as rootfs-skeleton and rootfs-packages can hold what really makes puppy a Puppy and then a package manager (apt-get, yum, pacman, slaptget, packdude) can offer full distro compatibility

[saintless]

Hi, Jamesbond.

I take the Ubuntu (or Debian) packages and build them up. The result is Puppy to the core, yet will work and is fully compatible with Ubuntu/Debian (I hope )

Unfortunately fully compatible is impossible in my opinion if you keep Puppy kernel, but the result will be much more Debian/Ubuntu repository compatible system than Dpup or Lucid.
Still this is very interesting project and it worth separate development. Is it possible to upload some ready for testing resulted iso?

What I see as a problem from quick thoughts:

1. Different system structure - for example rc.local is located different in Debian and Ubuntu and .xinitrc is .xsession. I know such small things seems not important but some packages in Debian or Ubuntu repository may and most probably will depend on them.

2. Forget about packages related to kernel downloaded from Ubuntu or Debian - linux-headers for example and maybe VirtualBox and similar programs.

3. I know it is not very welcome replacement here, but still... Forget about using systemd and programs depending on systemd from Debian or Ubuntu repository. With Debian/Ubuntu kernel and initrd.img you can boot with systemd by adding this to the boot code init=/bin/systemd With Puppy kernel and initrd.gz this option is lost forever.

4. Some programs from Debian/Ubuntu repository will not work without changes for root account.

5. Dpkg reads the information from /var/lib/dpkg/info , status, available. It is very easy to break dpkg database using different package managers as option to install pet or other distro packages. It should be dpkg (+apt-get or Synaptic) only or you are looking for troubles.

Toni

[mavrothal]

As far as I know all package managers allow for non-updatable packages.
Kernel can be a problem when it comes to drivers, but ubuntu at least is using pretty recent kernels and as long as the puppy kernel is the same version and has devtmpfs=y should not have issues, even if the user decides to go with the you-do-as-I-say systemd

[jamesbond]

Puppy does use a number of pets though not in woof and characteristic to puppy (jwmconfig for example).

Yup, I'm aware of that, though I don't know what exactly are needed.

However this could be fixed bu adding them flat into the recently added woof-CE/woof-code/rootfs-packages directory

Either that, or a collection of tarballs we can simply expand upon building Or if you want to be proper, make them into DEBs

Does you 300-liner considers rootfs-packages and if it does does it also runs any pinstall included there?

Yes.

This can be a really interesting development as rootfs-skeleton and rootfs-packages can hold what really makes puppy a Puppy and then a package manager (apt-get, yum, pacman, slaptget, packdude) can offer full distro compatibility

Yes, that's the plan.

Unfortunately fully compatible is impossible in my opinion if you keep Puppy kernel, but the result will be much more Debian/Ubuntu repository compatible system than Dpup or Lucid.

Thanks Toni, it's good observation. All of your observations are valid. You are right, the word "fully" must be qualified.

Still this is very interesting project and it worth separate development.

I'm posting this here in the hope of attracting current and future Woof developers (hopefully enticing the guys who want to do Woof but currently thinks that Woof is "too hard"). If there is enough interest I will start a new thread in Cutting Edge section

Is it possible to upload some ready for testing resulted iso?

The current script produces puppy.sfs, not iso, yet. But it is just a quantum leap from puppy.sfs to iso.

1. Different system structure - for example rc.local is located different in Debian and Ubuntu and .xinitrc is .xsession. I know such small things seems not important but some packages in Debian or Ubuntu repository may and most probably will depend on them.

I haven't checked them. I thought most distros have .xinitrc and .xsession in $HOME. Puppy does, too. In any case, going forward, we can align Puppy's structure to be more "standards"-compliant, so to speak, for compatibility reasons. That being the case, there is a limit to how far we can mold Puppy before it becomes not-Puppy so your concern is valid.

2. Forget about packages related to kernel downloaded from Ubuntu or Debian - linux-headers for example and maybe VirtualBox and similar programs.

Yes, in this case, we need to provide our own DEB repository. Aligning Puppy to work with standard kernel can be done but it's so far off I wouldn't even consider it at the time being.

3. I know it is not very welcome replacement here, but still... Forget about using systemd and programs depending on systemd from Debian or Ubuntu repository. With Debian/Ubuntu kernel and initrd.img you can boot with systemd by adding this to the boot code init=/bin/systemd With Puppy kernel and initrd.gz this option is lost forever.

Correct, although, going forward, anybody's who likes systemd can propose modification to Puppy core so that it can work with systemd.

4. Some programs from Debian/Ubuntu repWhat I see as a problem from quick thoughts:ository will not work without changes for root account.

Correct. I can't remember, I think dejan or whoever was planning to enable multi-user to Puppy and has committed some code too.

5. Dpkg reads the information from /var/lib/dpkg/info , status, available. It is very easy to break dpkg database using different package managers as option to install pet or other distro packages. It should be dpkg (+apt-get or Synaptic) only or you are looking for troubles.

No other package manager, it's dpkg (and its front-ends like apt-get/synaptic onwards). If you want to install pet, use a pet2deb converter and then install it with dpkg.

I guess going forward, we need to have names for these (right now, all these independent components are all called "Woof-CE" - which is super confusing.
a) The name of the repository that holds puppy-builder code, and puppy core code ("Woof repository").
b) Puppy builder code ("Woof build system").
c) Puppy core code (rootfs-skeleton, initrd-skeleton).

Please decide on what you want to call these three, but please don't call all of them them "Woof-CE". The script I'm talking about touches on point b, the puppy builder code (with minimal modification to puppy core code). Toni's concern are mainly for Puppy core (point c).

[saintless]

As far as I know all package managers allow for non-updatable packages.

If you install not dpkg registered package it should be very carefully configured.
Make sure not to overwrite existing executables and libs.
Very important because you will change them with different version that will probably not work for the package that installed the overwritten executable or lib.
And after uninstalling the same non-updatable package you will remove the executable or libs that were installed and overwritten already. The result is missing files and dpkg can't solve the problem because this files are actually manually removed and the related program does not work anymore, even dpkg says "All is fine here".

Kernel can be a problem when it comes to drivers, but ubuntu at least is using pretty recent kernels and as long as the puppy kernel is the same version and has devtmpfs=y should not have issues, even if the user decides to go with the you-do-as-I-say systemd

No. It will not work. Tested with official Debian kernel and porteus-initrd.img Even in this case systemd boot code does not work. No way to make it work with puppy kernel + puppy inird without changes.

Toni