Auto-build a Puppy iso; single script with optional gui

[rufwoof]

A nice simplification IMO for ease of use would be some guidance as to a appropriate choice of kernel to go with the choice of installation.

I found the basic woof-ce build instructions easy to follow/do once they were highlighted, but did hit a barrier when asked which of 24 or so different kernels to use. A default ENTER choice that perhaps picked the LTS/default Ubuntu or whatever default kernel for the build would have been nice given that it takes quite a while to run through the build and I had chosen the wrong choice (having just opted for a higher kernel version number that wasn't really compatible).

Might be best to even just restrict to either the Ubuntu (xenial now that tahr is EOL) or Slacko (Debian or Slackware main trunks) core Puppy Choices. Nice and easy for new users to perhaps install via pre-built ISO initially, but later "compile" (build) their own updated versions (security updates/upgrades). And then perhaps only going on to native woof-ce for other choices as their understanding develops.

[wiak]

A nice simplification IMO for ease of use would be some guidance as to a appropriate choice of kernel to go with the choice of installation.
...
Might be best to even just restrict to either the Ubuntu (xenial now that tahr is EOL) or Slacko (Debian or Slackware main trunks) core Puppy Choices

Yes, that is a good tactic rufwoof. Main reason I haven't made any simplification like that thus far is that I don't myself yet know the best combinations to offer...

Hopefully, experienced pup builders (such as Billtoo) will feedback (in detail) other combinations that work really well and then I can use that info to provide the best default no commandline argument results (aside from choice of maybe Debian Stretch, Ubuntu Xenial or latest Slackware based). An option switch or switches for more advanced users would be used to turn on more complete choice control for the user and an eventual optional gui to make things easier still for some. Hopefully no-one thinks I've released makepup just as a quick way to build the default distro it provides - that's just an easy exemplar. Rather, I've always hoped for people to try building other pups with it and feed back what best works.

One of the good things about makepup routines, I hope/think, is that they can also be used to modify default woof script build actions (in fact that's what they do), which may come in handy at times for those who like hacking existing scripts.

wiak

[wiak]

Just a quick try to build a Debian Dpup Stretch iso with:

Code: Select all

./makepup -d 1 -r 1

where --distro 1 (-d1) produces Debian-based pup distribution
and --release 1 (-r1) produces Stretch release

Using makepup default woof-CE-rationalise branch.

I forgot to choose a kernel so just used the makepup default kernel number 24, which turned out to be:

Code: Select all

# uname -a
Linux puppypc6641 4.1.38 #1 SMP Thu Mar 9 23:23:15 +08 2017 i686 GNU/Linux

EDIT: No... It wasn't that I 'forgot to choose a kernel', it's because for stretch, _00build.conf forces choice of kernel to 4.1.38 via included line:

Code: Select all

KERNEL_TARBALL_URL=http://smokey01.com/ttuuxxx/WoofCe/kernel-4.1.38-stretch/huge-4.1.38-stretch.tar.bz2

It built fine and booted fine and I'm posting from it now using the distribution-provided NetSurf browser. Only issue was the desktop icon for the webbrowser didn't work so I had to start netsurf-gtk from the commandline. I think there are a few more issues, but that's a woof-CE Debian Stretch branch issue, not makepup problem really.

Screenshot attached.

wiak

[rufwoof]

Nice.

Was that the default desktop layout? ... as it looks different to the more usual layout/arrangement (tray that spans the entire width of the display, and icons staggered in the top left sloping left-downwards).

[wiak]

Hi rufwoof,

makepup just runs the official woof build scripts and auto-answers some choices so basically the build result is the default stretch woof-CE provides. Whether any change to the auto-answers would change anything I don't know off the top of my head but doubt. What could be changed is to add a few extra packages or firmware (if available); choice to be made from those indicated as 'no' in DISTRO-PKGS-SPECS-'distro-version', the names of the extra ones wanted should be manually appended to makepup_extra.conf (using normal txt editor) in the form:

Code: Select all

<package_name>="true"

That package adding could conveniently be achieved by temporarily forcing the makepup auto-build script to pause at appropriate place by starting the makepup build command with option -p. During the pause, DISTRO-PKGS-SPECS-distro-version could be viewed to see what packages are available to be added and then makepup_extra.conf edited appropriately followed by unpausing/continuing makepup script (by pressing 'y' Enter):

Code: Select all

./makepup -d 1 -r 1 -p

could also change kernel used by also adding to the above makepup commandline:

Code: Select all

-H <kernel_number>

wiak

[davids45]

G'day wiak,

I replaced version 1 with 1.0.1 but left the previously created directories in place.
Ran makepup and it seemed to go through to the end. However terminal terminal (screenshot) does indicate something couldn't be done so the program exited?

However, I can't find any iso. The sandbox3 has no iso - if that's where it should be..

The files I'd expect in an expanded frugal iso are there in another directory, minus the main slacko sfs (screenshot).

Any advice?

Thanks,
David S.

[wiak]

G'day wiak,

I replaced version 1 with 1.0.1 but left the previously created directories in place.
Ran makepup and it seemed to go through to the end. However terminal terminal (screenshot) does indicate something couldn't be done so the program exited?

However, I can't find any iso. The sandbox3 has no iso - if that's where it should be..

The files I'd expect in an expanded frugal iso are there in another directory, minus the main slacko sfs (screenshot).

Hi David,

No, I have no idea why it didn't build for you. It worked perfectly before and should for you (unless something has been changed again on woof-CE github repo or you have used some makepup commandline options you haven't mentioned).

The most recent makepup version is only at 0.0.2, by the way, there is no version 1.0.1, but I presume you are actually using 0.0.2 in above test.

Could you kindly confirm you only used command:

Code: Select all

./makepup

and without any extra commandline options on that line. I haven't come across the issue you document, and report from BIlltoo the other day suggested 0.0.2 makepup worked for him too, but will try again now and see if all is well for me.

wiak

[wiak]

Hi David,

There doesn't seem to be any github changes to rationalise branch of late, so all should just work using command (without options):

Code: Select all

./makepup

The only thing is that I do usually always remove one directory before starting a new build: the woof-out_<distro-version> directory. I usually leave local-repositories directory intact and also leave woof-CE-rationalise (but makepup script itself by default automatically deletes that folder anyway).

I suggest you therefore completely remove woof-out_.... directory but should be okay to leave the others (though you can remove them all before trying again if you don't mind the longer download time).

I am in the middle of another test just now so haven't myself checked all is going well for me still, but I will later.

wiak

[wiak]

G'day wiak,

I replaced version 1 with 1.0.1 but left the previously created directories in place.
Ran makepup and it seemed to go through to the end. However terminal terminal (screenshot) does indicate something couldn't be done so the program exited?

However, I can't find any iso. The sandbox3 has no iso - if that's where it should be..

Hello again David,

I've just tried two rebuilds, one from empty directory (apart from makepup script itself being there) and one a retry where I didn't remove woof-out_ folders. Both builds worded fine for me.

I have two thoughts. Perhaps the woof-out_ directory you left was created from previous makepup 0.0.1 version, which used woof-CE-testing branch whereas makepup 0.0.2 uses woof-CE-rationalise branch by default and maybe that causes a problem (I can't verify that without a lot of re-tries!).

So better for you to delete woof-out_... directory before trying a rebuild using makepup version 0.0.2 again.

However, there is one other thing I notice. Your error messages indicate the woof build scripts were having a problem whilst doing a 'chroot' command. I believe that doesn't happen when I'm building because I'm building a 32bit target using 64bit Slacko64 as my host Puppy system - when the target is different architecture (32bit versus 64bit) I believe the woof build scripts don't attempt a chroot 'sanity test' (as they call it). I'd be surprised if running from 32-bit Puppy host (which I presume you are) is causing the problem but I'll have to test for that, though first I'll have to install a 32-bit Puppy on my own machine. If I can reproduce that error you are having I'll then have to fix it. However, I'm hoping that you deleting woof-out_... before running makepup works, in which case the issue is solved...

Please let me know if you manage that retry ((i.e. deleting woof-out... folder first).

wiak

[rufwoof]

Note well that you DO NOT need to remove the directory local-repositories on new tries of makepup. In fact, it is best not to since local-repositories contains cached the already previously downloaded packages and huge kernel. Keeping that saves bandwidth on later tries since they won't need to be downloaded again.

I believe even woof-out_* directory doesn't need to be deleted, though I don't know at this stage if leaving that dir speeds up the build process or not.

All should be working again now. Please let me know if any other problems surface. I'm working on next version now, which hopefully will use rationalised branch (in the hope damaging upstream woof-CE github commits won't be made into there until checked out thoroughly in testing branch).

Hi wiak.

Do you know whether if you leave local-repositories will the next run pull down any changed versions of those i.e. later versions of libs/programs if any are available?

I would like to track Xenial64 testing using your script as a form of rolling release i.e. I've set the script to point to testing by editing it to contain

WOOFBRANCH="woof-CE-testing"

and running it using

# Xenial 64 bit ubuntu with kernel 4.9.15 also builds devX
./makepup -t 3 -d 5 -r 28 -D

(as per your handy reference http://murga-linux.com/puppy/viewtopic. ... 543#965543)

with the intent that running that periodically will be like a puppy that is upgraded to all of the latest changes and security fixes.

Do you know whether woof-ce is automated to pick up the latest versions of Ubuntu/Xenial changes i.e. if they change xxxx.v1 to xxxx.v2 will that automatically be reflected into woof-ce or does it require some manual intervention by the woof-ce maintainers to adjust its database to point to the later version?

My thinking is that once built to open the new iso and drop the new puppy sfs and zdrv sfs (and devx sfs) in as direct replacements to existing versions into my frugal Xenial boot folder and reboot ... so its upgraded for the latest enhancements/fixes/security patches.

It would be nice to have the additional feature of a relatively simple means to track a rolling/updated version of Puppy (takes less than a hour to build from a completely empty folder (just makepup script in that folder) on my setup and used around <6GB of disk space).

[rufwoof]

Booting that build and it didn't like vmlinuz. Swapping that out for my existing one and it booted OK ... to what looks to be like the new style default desktop layout (that looks somewhat like xfce i.e. a central bottom panel rather than one that spans the entire width of the screen).

Working to a degree, but not functional (no firewall option available, try to change resolution and it black-screens ...etc). But as far as the script went everything seemed OK. It did build a 7.0.8.4 Xenial iso whereas my last manual Xenial build built a 7.0.8.5-efi iso so I'm not sure why it went for a earlier non uefi version (perhaps again something down to woof-ce rather than the script ???).

[wiak]

perhaps again something down to woof-ce rather than the script ???.

Almost certainly (until shown/proved otherwise )

I think that Debian Stretch woof-CE stuff was started by ttuuxxx with contributions by others (musher maybe, belham2) but from reading that thread there were some 'disagreements' and I don't the final woof-CE stuff was finalised (indeed much of the work seemed to be done as remasters with only initial build from woof-CE). I think a lot of that has been going on with woof-CE builders (i.e. a lot of the work still being completed by remastering). Better, I think, if they learned how to complete the work on woof-CE itself and then builds made by that could be complete and already polished. Also that would help with woof-CE development and bug-fixing. It is a pretty good system though - specially when the possibility of using stock Debian kernels and so on is completed (code for that started but not yet integrated into the build system).

wiak

[wiak]

Do you know whether if you leave local-repositories will the next run pull down any changed versions of those i.e. later versions of libs/programs if any are available?
...

with the intent that running that periodically will be like a puppy that is upgraded to all of the latest changes and security fixes.

Do you know whether woof-ce is automated to pick up the latest versions of Ubuntu/Xenial changes i.e. if they change xxxx.v1 to xxxx.v2 will that automatically be reflected into woof-ce or does it require some manual intervention by the woof-ce maintainers to adjust its database to point to the later version?

Hi again rufwoof,

No, I don't know, but I'm also interested in knowing the above, and for similar reasons, so I'll try and take some time to look into that especially since I'm becoming familiar with some of the woof-CE structure/scripts now.

May take a while before I know more though, which is one of the reasons I'm not working towards a further 0.0.3 makepup release yet - worth sorting out a few of these details first.

wiak

[wiak]

I've set the script to point to testing by editing it to contain

WOOFBRANCH="woof-CE-testing"

and running it using

# Xenial 64 bit ubuntu with kernel 4.9.15 also builds devX
./makepup -t 3 -d 5 -r 28 -D

(as per your handy reference http://murga-linux.com/puppy/viewtopic. ... 543#965543)

rufwoof, I trust you know, you could alternatively have included option:

Code: Select all

-w woof-CE-testing

on the commandline to point to that branch? Or alternatively to edit makepup.conf to do that instead?

All the current makepup options can be listed with:

Code: Select all

./makepup --help (or makepup -h)

Any of these methods work of course but for the released version of makepup I'll leave default script not pointing at testing branch so that default builds are more likely to work (makes no difference usually to the iso produced [since the distro-version builders haven't been making further commits for a longtime I think] - only to the internal woof build scripts which are being refined over time: mainly it seems by jlst).

wiak

[wiak]

G'day wiak,

I replaced version 1 with 1.0.1 but left the previously created directories in place.
Ran makepup and it seemed to go through to the end. However terminal terminal (screenshot) does indicate something couldn't be done so the program exited?

Hello again David,

I have now tried running makepup from a 32bit Pup (current slacko32 ver 6.3.2) to build a 32bit (x86) target, the default provided by simply using command:

Code: Select all

./makepup

with no optional arguments.

I've tried that from both an empty directory (aside from the makepup script itself) and also as a second run leaving all the previously created directories in place. Both runs completed successfully. So, sorry, I can't understand thus far why you are getting those error messages - all working fine for me, and seemingly for others. If you have the chance to try on another computer or maybe use a different Puppy as the host build system is all I can suggest off the top of my head at the moment. What Puppy are you running the script from at the moment? woof-CE documentation itself suggests you should use a reasonably current one. You do NOT need the devx loaded since makepup downloads a zip file of woof-CE build environment and that is enough to build the iso and frugal files (EDIT: however, you may sometimes get a smaller iso if build done with devx loaded on host - depends if any binaries could be stripped - the strip program is in devx).

Yes, once the build is complete, the frugal files for booting should be found in sandbox3/build and the iso (assuming the default makepup build) currently in woof-output-slacko-6.9.9.9

Tomorrow I plan to try making a build with makepup on my old Pentium M, 1 GB ram non-PAE laptop system (have to locate it from the bottom of my cupboard somewhere first though...).

wiak

[rufwoof]

Thanks wiak

[peebee]

Do you know whether if you leave local-repositories will the next run pull down any changed versions of those i.e. later versions of libs/programs if any are available?

Yes - latest versions will be downloaded by ./1download as long as repositories have been updated by ./0setup....

./2createpackages can then be run selectively on just the updated packages.

This makes building an updated system very fast (c. 15mins for LxPupSc).

[rufwoof]

Thanks peebee.

Just built a 64 bit Xenial from woofce-rationalise (I hard coded makepup values to ...

Code: Select all

# Default config:
nTARGETARCH="3"              # i.e. target is x86 (1=arm, 2=x86, 3=x86_64)
nCOMPATDISTRO="5"            # i.e. distro is slackware (1=debian, 2=devuan, 3=slackware, 4=trisquel, 5=ubuntu)
nCOMPATVERSION="2"           # 2 is xenial (1 trusty)
WOOFBRANCH="woof-CE-rationalise" # woofbranch, e.g. woof-CE-testing
#WOOFBRANCH="woof-CE-testing"
nHUGEKERNEL="28"             # i.e. hugekernel to use huge-4.9.15-xenialpup64.tar.bz2
interactive="false"          # "true" means: Require user input during the build
gui="false"                  # "true" means: Use gui frontend (wiak remove comment when done: gui not yet implemented)
keep="false"                 # "true" means: Keep previous woof download files and directories
pause="false"                 # "true" means: pause makepup script just after merge2out routine to allow package selection changes
DEVX="true"                 # "true" means: Create DEVX sfs

Built and booted fine.

Sound, internet, change resolution, firewall, savefolder ...etc. all working as expected. Only problems I've found so far is that the drives top right unmount hot area doesn't work (just opens the drive in rox) and using pupclockset to change the trays time format to a custom %a %d %b %R ... and then saving, had the top of screen panel/tray lose its settings to leave just setup and jwmdesk icons only in that panel ... and no minimised windows shown either (I'm having to use Ctrl-Alt-Tab to switch between windows as I post this).

[wiak]

Yes - latest versions will be downloaded by ./1download as long as repositories have been updated by ./0setup....

./2createpackages can then be run selectively on just the updated packages.

This makes building an updated system very fast (c. 15mins for LxPupSc).

He Peebee, rufwoof and anyone reading,

Thanks for that info, Peebee. It's especially useful to have such input from an experience Puppy builder like yourself. I can easily add a --update/-u switch to makepup to modify its behaviour for update packages use. Is there a way provided by 1download to indicate which packages have been updated such that I can automate 2createpackages to selectively operate on just these packages in terms of update? A step by step example of how you go about such an update, including how you use 2createpackages to selectively operate would help me automate these mechanisms via makepup function.

Otherwise I can use modification times to local-repository files as a possible mechanism but still need to know how to use 2createpackages for selective operation.

Such modifications to makepup are likely to be made later anyway since other improvements still in progress.

wiak

[rufwoof]

I've just run makepup for a second time, leaving everything from the first run (that took about a hour) as-is, and the second run took 48 minutes (and seems to have built the iso and devx ok). Seems about right as I'm on a 100Mbs link so I guess around 12 minutes of download time for the first run not being required in that second run.