Auto-build a Puppy iso; single script with optional gui

[wiak]

The set on ibiblio is getting a bit "long in the tooth" now but there are kernels in many places with links available on the forum - I don't know if wiak's tool is able to make use of these or not

Yes there is a button on the gui that opens a folder that the user simply needs to drop a huge kernel into. makepup looks after later transferring that kernel into the huge kernels directory peebee mentions.

wiak

[wanderer]

hi wiak

just a question

could your script be used to build (as a default setting)
a super tiny woof-ce puppy
with just the absolute minimum to start
that is very fast to build

I think a lot of people
would like to start out very simple
so that it would be easier for them to learn the woof-ce system

the 2 biggests impediments for me were
complexity
and the time it took to build each iso

wanderer

[nosystemdthanks]

the 2 biggests impediments for me were
complexity
and the time it took to build each iso

yeah i gave up on woof-ce rather quickly. it wasnt just the complexity, i was also being lazy. thats why i think this is such a great idea.

whenever its possible to have sane defaults that allow the inexperienced to "just run it and get a result" before learning how to to tweak the results, i think youve got a winner.

could your script be used to build (as a default setting)
a super tiny woof-ce puppy
with just the absolute minimum to start
that is very fast to build

I think a lot of people
would like to start out very simple

i would be very tempted to use the settings for minimal if they were available. however, i think more people would want "full" or at least "basic" (more than minimal) to be the default.

if you go this route, you dont technically need to have "profiles" as a feature. but imagine if you had that.

imagine if you built into this, a variety of profiles that you could select very easily-- and optionally-- by name:

basic (medium)
minimal
full (dvd required?)

i mean this could become a very exciting new way to build puppy, where instead of making an iso you could just write a profile and then put it on the forum (or pastebin.)

but you wouldnt have to write a profile unless you wanted to-- because you could select one of the profiles thats included.

but you wouldnt have to do that either unless you wanted to-- because wiak would choose whatever profile was the best default, unless you selected one.

if i had designed everything about puppy (years later, after all the hard work was already completed) this is how i would do it. note that in the late 1980s or early 90s i said "it would be so cool if we could have programmable cpus that could behave like a motorola or intel chip."

i didnt think it was possible (commercially) but we have those chips now. and i think the future is programmable distro building. you can share iso files, but you can also share a text file that says how the iso should be built.

set it with all the effort of starting the wash, and come back to a brand new iso file. wave of the future-- but partly inspired by woof-ce for sure.

i mean everything ive done to remaster was inspired by woof-ce, actually. wiaks script is even closer to what i was thinking. so yeah i will throw these ideas around, but i dont expect wiak to use them. very cool if he does!

i first proposed the idea of a small/medium/large puppy about 10 years ago; today i could actually produce it. back then, sfs was magic and (i was quite the noob) losetup was like the dark arts.

but just based on this thread, i believe wiak could do it better than i would. and like i said: im lazy. the reason i made my remaster script automatic is that i dont want to go to the trouble of remastering the usual way. this communitys attitude towards remastering (and fighting unwanted complexity) is the primary attraction for me. i dont think any distro community (ever) is more devoted to remixing.

one of the coolest things about profile-based building would be its ability to bring back older, favourite versions. obviously no profile would perfectly recreate every aspect of an older version. for nostalgia, for demonstration, for more practical uses even, it would make it easier to maintain and approximate your favourite older versions of puppy.

i was a fan of puppy 2.x. one of the features was the 2.x kernel. if i made a profile for that version, it would probably not use the same kernel, it would use a newer one (and lose one of the advantages of that older version. but gain some new advantages.) but if the goal was "an update of good old 2.x" that would be much more possible and practical this way.

[wanderer]

yeah profiles

that's a great idea


wanderer

[nosystemdthanks]

it sounds really complicated, but how we would get profiles isnt:

1. first hook a few options in the existing script to make it so you can have a "minimal" and "regular" build just like you want.

2. wiak can pick whether regular or minimal is the default profile, so if one isnt selected then you get the default.

3. make it you can choose either profile from the gui or without it.

4. gradually add features that each profile can use. like maybe you want the profile to be able to select wm and wallpaper. so you add those.

5. i wrote a program language with nearly a 100 commands this way. just adding a couple features at a time, youd be amazed what it can become. still, no one is saying wiak has to do this.

[aaaaa]

Profiles for ISOS, multi user, or at least a usable non root user, etc.

Good ideas.

But there's a little problem. Implementing anything in woofce or any other woof is incredibly difficult.

It not only requires devoted developer(s), but also devoted testers, and that's more than a luxury nowadays.

However the project is slowly getting rid of many things that prevent further improvements. There's a pull request to delete ~500 files.

The problem is the design itself inherited from the original woof.. yes that's the main problem.

Just fixing the current bugs is a huge task. Some other changes were also implemented but the code was lost because there was no one to test them, many changes happened, and it's a pain to rebase all of that work.

My advice is as usual see other options.., i see TazPup, which is small, mistfire only takes a few puppy-specific things and tries to blend 2 distros. clever. or the debiandogs if you want a completely working package manager, etc..

However woofce still have the ability to use any distro and a very small iso. But you'll see it in action after a massive earthquake...

[nosystemdthanks]

But there's a little problem. Implementing anything in woofce or any other woof is incredibly difficult.

which is why i gave up on woof immediately and went in a different direction trying to find a shortcut. i ended up with a distro instead, but the fact is that if i were getting paid to implement this stuff on top of woof (not saying it would or should happen, id rather do things my own way for free) id still insist on using my own tools to interface with it.

and that would mean youre stuck with python, which woof doesnt require.

but then without my tools i never would have created a distro. i mean i can code in bash, i implemented a simple logo interpreter in 100 loc of bash. but my bash skills arent up to what woof needs.

It not only requires devoted developer(s), but also devoted testers, and that's more than a luxury nowadays.

yes. id much rather try wiaks version than try out woof again. im not trying to be unkind. i tried woof because people insisted, and i couldnt really comment on it unless i tried it, right?

and then i set out to do something automatic, which wiak has done a far better job with-- while i do something else.

However the project is slowly getting rid of many things that prevent further improvements. There's a pull request to delete ~500 files.

nice work.

The problem is the design itself inherited from the original woof.. yes that's the main problem.

i agree, but its safer for you to say. you also have a much better idea what youre talking about, so its safer and smarter if you say it.

many changes happened, and it's a pain to rebase all of that work.

if it wasnt, i might have tried it.

My advice is as usual see other options.., i see TazPup, which is small, mistfire only takes a few puppy-specific things and tries to blend 2 distros. clever. or the debiandogs if you want a completely working package manager, etc..

or create a script that downloads tahr and devuan live, and create a hybrid iso with a puppy mode that boots into puppy with some files from devuan, or boots into devuan mode with some files (like petget) from tahr. which is the silly (but fun and educational) approach i took. note that i now use that distro all the time, but its no longer puppy-based. the script that builds it came out of working with puppy though.

but the point of course is that we get to choose from many different solutions. the advantage of wiaks of course is that it is based directly on woof-ce.

However woofce still have the ability to use any distro and a very small iso.

which is pretty cool.

whether to rebuild/modify a complex thing like woof or try from a different direction is going to depend on how much of a perfectionist the developer is.

because the person that goes to the trouble of fixing the complicated thing is usually the one who cares most about perfection. and compatibility, whatever the cost. values we can at least admire and salute. they keep puppy more like puppy, and thats in demand.

[wiak]

Well maybe I'll get back to this sometime. I'm not sure. I was simply in 'maintenance' mode with it because I've been working on so many other things recently (too many really) and only had enough time and energy (almost) for that. Tinkering with woof-CE via script interventions is quite tricky unfortunately (though more could certainly be done - anything being possible...) - better/easier though if the build system moved towards a debootstrap style of operation (such as used in fredx181's mklive-stretch offering). But that's a big ask in terms of coding complexity and woof-CE re-write, which I'm not involved in anyway.

I had hoped to get back to IUP/Lua code writing, since I also want to learn something different and I'm interested in Lua and found IUP rather nice.

I also want to take a look at EasyOS to see what that is all about; it looks quite interesting from a different perspective point of view.

Then, I also wanted to put my feet up, though it is gratifying that a fair number found makepup somewhat useful at least as an introduction to woof-CE itself.

wiak

[aaaaa]

A big thanks to wiak for this effort.

Getting into woofce might also mean trouble if you want to edit stuff or keep custom configs, as woofce is going through changes that also affect the list of pkgs to be added. Your stuff might end up producing a broken build.

Well, there's no other way to improve things, woofce is a collection of distros, and important changes have to be enforced in all of them. Consistency has a huge price as well. As i said, i also lost many codelines

Flavors, people don't want to be forced to use a certain DE, but in woofce you are AHAHA! I'm cheating using something other than the standard and enforcing in my build. That's life i guess...

[wanderer]

yes aaaaa and everyone

that was just a question
I think I will go back to playing with woof-ce
and just try to make a tiny iso
the woof-ce team will continue to refine things
(and they could use the next puppy development subforum)

woof-ce is the core of puppy
though there are many branches

a big thanks is in order to everyone that has made puppy great

barry k
john murga the forum
the woof-ce team
wiak ...
etc etc etc
too many to list

wanderer

[nosystemdthanks]

Flavors, people don't want to be forced to use a certain DE, but in woofce you are AHAHA! I'm cheating using something other than the standard and enforcing in my build. That's life i guess...

i really (actually) dont want to take over this thread, but i will add this:

the easiest way to add profiles to this wonderful script of wiaks would be to add a simple hook for other tools to run-- whether based on python or bash or lua or whatever-- either before mksquashfs runs or even after wiaks script completes the iso.

then no matter what goes on with woof, you can "unleash" (sorry, unleash is the name of one of barrys things, im just making the obvious pun) the second tool on it, which can run automatically from wiaks script. and if its not found, it simply wont run.

the disadvantage is that it wont make it run faster unless we go with wanderers default minimal profile.

i mean if the goal is to have the whole process go faster, the first stage (wiaks) has to be faster by default before it calls something else. and then the second stage for profiles could actually be more than one tool (default: no second stage, no worries) maintained by one or more different people with entirely different goals. these are ideas, not requests.

[aaaaa]

i really (actually) dont want to take over this thread, but i will add this:

the easiest way to add profiles to this wonderful script of wiaks would be to add a simple hook for other tools to run-- whether based on python or bash or lua or whatever-- either before mksquashfs runs or even after wiaks script completes the iso.

We're talking about the PKGS_SPECS.

I tested the script and it does work.

I changed this line
if [ "$KEEPBRANCH" = "false" ]; then

to
if [ "$KEEPBRANCH" = "false" -o ! -f ${woofclonefile} ]; then

You could just select all the packages you want and then puppy automatically resolves dependencies and all, but that's not the case.

The way things work now make profiles a tedious task, you have to do everything manually.

The advantages of this system allows cross-builds, and all sorts of weird stuff that is practically impossible in other build systems, but a high price.

To edit every part of the system you must be highly skilled, and in woofce there is usually only 1 person doing everything or nobody working at all, and the package management was not my priority because i see puppy more like a rescue distro. But i'm editing stuff to make things faster, including the ppm. This stuff needs a small redesign.

Regarding the auto-build script, i see it can be improved and be part of the git repo, but when it comes to my plans, it doesn't involve the forum..

[nosystemdthanks]

in woofce there is usually only 1 person doing everything or nobody working at all, and the package management was not my priority

all understandable. theyre probably afraid to step on toes.

but its typical for almost every free software project to have one person working on it at a time. realtime collaboration is a rare exception, at least most free software has one developer, even when its good.

[davids45]

G'day,

Still trying to build a BionicPup with makepup-1.5 but can only get a XenialPup.

I can 'select' the kernel by putting it in the kernel2add directory and leaving the makepup.conf kernel line as "". (As below).

But what entry (and where) is required to pick a Ubuntu version? Looking at the makepup script I see:

# Default config:
nTARGETARCH="2" # i.e. target is x86 (1=arm, 2=x86, 3=x86_64)
nCOMPATDISTRO="5" # i.e. distro is slackware (1=debian, 2=devuan, 3=slackware, 4=trisquel, 5=ubuntu)
nCOMPATVERSION="3" # i.e. release version is 14.2 (1=14.0, 2=14.1, 3=14.2)
WOOFBRANCH="woof-CE-testing" # woofbranch, e.g. woof-CE-testing
nHUGEKERNEL="" # i.e. hugekernel to use is huge-4.4.70-s32-700_PAE
INTERACTIVE="false" # "true" means: Require user input during the build
gui="false" # "true" means: Use gui frontend
KEEPBRANCH="true" # "true" means: Keep previous woofbranch zip-file and woof-CE-branch directory
PAUSE="false" # "true" means: pause makepup script just after merge2out routine to allow package selection changes
DEVX="false" # "true" means: Create DEVX sfs
REBUILDALL="false" # i.e. rebuild changed packages only. Alternative, for now, is "true" for ALL_PACKAGES.
POPUPTHEMES="false" # "true" means: Pop-up 'choose themes' gui during the final 3builddistro-Z process
POPUPEXTRA="false" # "true" means: Pop-up 'choose extra packages' gui during the final 3builddistro-Z process
EXTRACONF="true" # "true" means: use makepup_extra.conf, if exists, to overwrite rootfs-packages.conf extra packages.
ADDPETS="false" # "true" means: adds to build any dotpets you have dropped into folder local-repositories/pets2add/
ADDPKGS="false" # "true" means: adds to build any distro-compatible packages you have dropped into folder local-repositories/pkgs2add/

I'm thinking, like in Slackware, it's the number I put in the nCOMPATVERSION line that would select Tahr, Xenial or Bionic but no luck so far trying '3' for a Bionic - being the third and latest Ubuntu version?

My last-tried config file was:

nTARGETARCH="2"
nCOMPATDISTRO="5"
nCOMPATVERSION="3"
WOOFBRANCH="woof-CE-testing"
DEVX="false"
INTERACTIVE="false"
gui="false"
KEEPBRANCH="false"
nHUGEKERNEL=""
PAUSE="false"
REBUILDALL="false"
POPUPTHEMES="false"
POPUPEXTRA="false"
ADDPETS="true"
ADDPKGS="false"

but gave me a Xenial iso output (by its name - I haven't actually installed it to confirm ).

And I haven't tried '4' here in case there's also Precise in the Ubuntu list of versions, so Bionic could be '4'?.

David S.

[peebee]

1 arm
2 x86
3 x86_64
Type number of target architecture: 2
...ok, x86

Woof builds a Puppy based on the binary packages from another distro.
We sometimes refer to this as the "compat-distro".

1 debian
2 devuan
3 slackware
4 ubuntu
Type number of compat-distro: 4
...ok, ubuntu

The compat-distro usually has release versions
Choose which release you want to obtain the binary packages from.

1 trusty
2 upupbb
3 xenial
Type number of release: 2

[wiak]

Hi David and peebee,

I'm not on a Puppy at the moment. If I was, easiest is just to run makpup in gui mode with 'makepup -g' and the drop down will tell you which numbers to use for which distributions for those using commandline mode such as yourself david. However, I think it is:

Code: Select all

-t --target (TARGETARCH): 1 is arm; 2 is x86; 3 is x86_64

-d --distro (COMPATDISTRO): 1 is debian; 2 is devuan; 3 is slackware; 4 is trisquel; 5 is ubuntu

-r --release (COMPATVERSION): 1 is UbuntuPupBB32 or UbuntuTahr64; 2 is UbuntuTahr32 or UbuntuXenial64; 3 is UbuntuXenial32

In which case, for UbuntuPupBB32 you would want (if I've coded correctly? - which is looking doubtful...):

Code: Select all

-t 2 -d 5 -r 1

Or here are the code lines that I use in makepup to find the different combinations:

Code: Select all

    <text><label>$(gettext 'target architecture [-t]: ')</label></text>
	<combobox>
     <variable>TARGETARCH</variable>
     $(combobox_list 2:x86 1:arm 2:x86 3:x86_64)
    </combobox>

    <text><label>$(gettext 'distro base [-d]: ')</label></text>
    <combobox>  
     <variable>COMPATDISTRO</variable>
     $(combobox_list 3:slackware 1:debian 2:devuan 3:slackware 4:trisquel 5:ubuntu)
    </combobox>

    <text><label>$(gettext 'release version [-r]: ')</label></text>
    <combobox>  
     <variable>COMPATVERSION</variable>
     $(combobox_list 3:Slackware14.2 1:DebianStretch 1:DevuanAscii 1:Slackware14.0 1:UbuntuPupBB32 1:UbuntuTahr64 1:TrisquelBelenos 2:Slackware14.1 2:UbuntuTahr32 2:UbuntuXenial64 3:Slackware14.2 3:UbuntuXenial32)
    </combobox>

However, when I upgraded makepup to hopefully find UbuntuPupBB32, I didn't really test it because was in middle of other work, but relied on feedback to tell me if I got it wrong. According to what peebee has written in post above, I seem to have indeed got the code in makepup gui wrong, but my system has run out of space so I have no Puppy on here to test, so please let me know if I have to change the combinations in makepup code and I'll fix it...

EDIT: Just had a look at woof-CE on github and I note that trisquel seems to have been removed, which would change the number ordering. Alas, whoever makes these numeric changes didn't inform me and that ordering is of course required by makepup. Please let me know what current numberings are and I'll alter makepup accordingly (looks like ubuntu in now --distro 4, or maybe trisquel is still there and I've just not noticed it - been a while since I checked).

EDIT: Looks like -t 2 -d 4 -r 2 might indeed be correct and I'll have to modify makepup accordingly. If so, I'll install a pup to a usb in the next day or less and do a quick manual woof-CE puppy build to check the current numeric values needed and modify makepup accordingly - sorry about that...

wiak

[davids45]

G'day wiak & peebee,

Thanks for your quick and informative replies about my number choices in makepup's config not giving me a BionicPup.

I'll try 2-4-2 and if that's no good, maybe 2-4-1.

Fairly quickly if I watch what's happening on my monitor, I can see the basic Pup being built - if I don't like it, I can stop makepup. Last night I tried 2-5-4 and could see it was still a Xenial Pup being made, so just killed makepup on the terminal, shut down the computer, and went to bed.

Once I get the right Bionic settings, I can try removing default packages I don't want and adding in my preferred ones in one form or another, plus add config files and the like (as .pets) for my desktop.

Then I can try switching kernels to maybe improve boot speed and general Puppy operation.
Incomplete pinboard display during booting is my main new-Puppy bugbear at present, fixed with clicking a pinboard icon linked to a script to restart-X. Maybe I'm loading too many sfs files (with pinboard icons) and have too many partitions (lately, these are very slow to display across the pinboard).

Anyway, makepup should let me try options to improve my particular/peculiar situation (if possible).

Thanks both again.

David S.

[wiak]

Hello David,

Yes, the loss of trisquel changed all the numbers so I'll have to upload a different makepup version. In the meantime, post 2 of this thread (at top) gives some examples of current numeric values that should sort your problems out. Meanwhile, I'll fix the numbers in next release for makepup.

Current numeric values examples in post 2 of this thread here:

http://murga-linux.com/puppy/viewtopic. ... 543#965543

So, currently, for 32bit upupbb, you do indeed need -t 2 -d 4 -r 2

wiak

[wiak]

makepup version 0.1.6 uploaded.

To first post of this thread, here:

http://murga-linux.com/puppy/viewtopic. ... 541#965541

Following Trisquel selection being removed in woof-CE, this new makepup version hopefully fixes numeric values for distribution selection, but needs testing, thanks.

uPupBB32 is currently the default build (i.e. -t 2 -d 4 -r 2)

But please note the following post from PeeBee regarding kernels for uPupBB32:

http://murga-linux.com/puppy/viewtopic. ... 107#994107

EDIT: Note that you do get one popup window during uPupBB32 build (and maybe any Ubuntu?); I can't remember why sorry or if you can just ignore it (please let me know), though I think someone drew attention to this earlier in the thread somewhere... I'd have to study the woof-CE scripts again to remove that popup and don't have time at the moment, sorry. EDIT2: I think now that the popup just vanishes of its own accord if ignored.

EDIT2: I wouldn't recommend building a pup on a usb stick via usb2 interface - that takes forever (very slow), but fine on a hard drive partition in my experience.

wiak

[davids45]

G'day wiak,

Thanks for taking the time to update makepup to 0.1.6. I'll give it a go or two later today.

Regarding the kernel to be used, is it still true to your recollection that putting the 'huge-kernel' tar.bz desired into the /local-repositories/kernel2add directory will override any other kernel in the config file (or will if I delete any kernel number in the config file line about the kernel to use)?

David S.