Auto-build a Puppy iso; single script with optional gui

A home for all kinds of Puppy related projects
Post Reply
Message
Author
User avatar
davids45
Posts: 1326
Joined: Sun 26 Nov 2006, 23:33
Location: Chatswood, NSW

huge kernel missing?

#16 Post by davids45 »

G'day wiak (and Billtoo),

Well, I think I started with an empty directory with just the make-pup script in it. I could be wrong :oops: .

The ERROR that stopped my first trial seemed (to me, reading the terminal window at the end) to be it couldn't find/download the huge-kernel.

Billtoo's post about getting a working makepup-made Pup suggests all he did was put the missing Slacko huge-kernel in the directory as a .tar, and then did something/somewhere to finish the iso creation process?

I couldn't find the "woof-CE huge_kernel.sh" mentioned in your last post that may be the something/somewhere he did or ran?

In my archive of Puppy stuff, I do have a couple of old huge-kernels from playing around with up-dating Tahr Pups - but no old or new Slacko huge kernels.

The EMPTYDIRMAKER text file in my huge_kernel directory seems empty (when I open it in a text editor) and there is nothing else in that huge_kernel directory.

As a believer in hybrids (is my Prius showing? :D ), if I copied a Tahr huge_kernel tar into the huge_kernel directory, would that work to give me a hybrid iso from what's already downloaded, if I was told where to find the script to do so?

David S.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#17 Post by wiak »

Hi David,

I found the problem, I believe. One of the main woof-CE build scripts merge2out was modified by someone two days ago and they seem to have not tested it was still working with downloading kernels before committing it to github. (EDIT: Please don't see that as a criticism - these are human errors inherent in any coding - I do the same all the time - I even upload dotpets only to discover later the last change I made broke everything...)

Yes, Billto's method works round that issue.

Please see below for my comment on relevant merge2out committ woof-CE github. Sorry for all the time wasted. I should probably not use testing branch of woof-CE but other branches may be too old to be interesting... EDIT: decided to keep testing branch as default - previous issues were my own fault for not keeping closer eye on woof-CE developments whilst I was too focussed on makepup code writing ;-)

wiak
Last edited by wiak on Tue 12 Sep 2017, 05:01, edited 2 times in total.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#18 Post by wiak »

place-marked

wiak
Last edited by wiak on Tue 05 Sep 2017, 23:12, edited 1 time in total.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#19 Post by wiak »

place-marked

wiak
Last edited by wiak on Tue 05 Sep 2017, 23:13, edited 1 time in total.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#20 Post by wiak »

@David:

Note well that you DO NOT need to remove the directory local-repositories on new tries of makepup. In fact, it is best not to since local-repositories contains cached the already previously downloaded packages and huge kernel. Keeping that saves bandwidth on later tries since they won't need to be downloaded again.

I believe even woof-out_* directory doesn't need to be deleted, though I don't know at this stage if leaving that dir speeds up the build process or not.

All should be working again now. Please let me know if any other problems surface.
EDIT: using testing branch again but will endeavour to monitor upstream woof-CE changes to keep makepup up-to-date with these.

wiak
Last edited by wiak on Tue 12 Sep 2017, 04:59, edited 1 time in total.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#21 Post by wiak »

makepup version 0.0.2 uploaded.

Changes:

# Default config now uses woof-CE-rationalise rather than woof-CE-testing
# Now auto-creates config files and uses them. Removed options --makeconfig and --config since no longer required.
# makepup commandline options now override the hard-coded defaults and/or those in the makepup.conf file.
# Added -p/--pause option to pause script to allow editing of makepup_extra.conf for adding extra packages available from woof-CE

Code: Select all

./makepup -p [any other optional args you like]
The script will continue on to completion from that stage but you can exit it at that time, for examining the config files (makepup_extra.conf and DISTRO_PKGS_SPECS-distro-version) in a text editor, if you wish.

For details, check usage with command:

Code: Select all

./makepup -h
If you also want a devx squashfile to be created, use option --DEVX (or -D) on the commandline (note it is capital D).

wiak
Last edited by wiak on Tue 05 Sep 2017, 23:14, edited 1 time in total.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#22 Post by rufwoof »

Hi wiak

A version that built from Master would be useful i.e. built from the most stable/release version (if that is correct, I really don't understand WoofCE). That way anyone could in effect "update" their Puppy version ... run the script ... that pulls in updated/fixed versions of packages from Ubuntu (or wherever) along with any Puppy fixes made between when last built and the new build.

User avatar
Billtoo
Posts: 3720
Joined: Tue 07 Apr 2009, 13:47
Location: Ontario Canada

Auto-build woof-CE Puppy using single script

#23 Post by Billtoo »

Hi,

I gave the new version a try and it seems to be working well, I used
"makepup -D" to run the script this time and it produced the devx.

I put my preferred kernel in the huge kernel directory while the script
was running so as not to get the default kernel.

Will install the new iso shortly and see how it goes.

Thanks.
####################################################################
EDIT: I installed the iso to a 32gb SDHC card, it's plugged into a usb
card reader:
# inxi -bw
System: Host: puppypc7079 Kernel: 4.4.84-pae i686 (32 bit) Desktop: JWM 2.3.2 Distro: Slacko Puppy 6.9.9.9
Machine: Device: desktop System: ACER product: Aspire M5620 v: R01-A4 serial: PTS860X0348050CF642700
Mobo: ACER model: G33T-AM v: 1.0 serial: 00000000 BIOS: American Megatrends v: R01-A4 date: 12/19/2007
CPU: Quad core Intel Core2 Quad Q6600 (-MCP-) speed/max: 1603/2403 MHz
Graphics: Card: NVIDIA GF108 [GeForce GT 430]
Display Server: X.org 1.18.3 driver: nvidia tty size: 120x46 Advanced Data: N/A for root
Network: Card: Intel 82566DC-2 Gigabit Network Connection driver: e1000e
Drives: HDD Total Size: 32.1GB (14.3% used)
Weather: Conditions: 72 F (22 C) - Overcast Time: August 28, 12:26 PM EDT
Info: Processes: 186 Uptime: 12 min Memory: 294.6/8097.6MB Client: Shell (bash) inxi: 2.3.8
#
I installed some applications with PPM + compiled a couple, also
compiled the proprietary nvidia driver.
Attachments
screenshot.jpg
(92.61 KiB) Downloaded 753 times
makepupV2.jpg
(162.37 KiB) Downloaded 812 times

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#24 Post by wiak »

rufwoof wrote:Hi wiak

A version that built from Master would be useful i.e. built from the most stable/release version (if that is correct, I really don't understand WoofCE). That way anyone could in effect "update" their Puppy version ... run the script ... that pulls in updated/fixed versions of packages from Ubuntu (or wherever) along with any Puppy fixes made between when last built and the new build.
To build an iso from the master branch, should just need to use commandline argument:

Code: Select all

./makepup -w woof-CE-master
though, whether it will build good iso depends on upstream woof-CE master branch build state. Probably ok though. Aside from testing branch being exactly that, and thus unstable, I don't really know the differences in them all. Would be good indeed if that could be defined by someone who knows.

I wouldn't say it's like apt-get update though, more like a complete new debootstrap each time (though with all the major Puppy-provided apps and utils bolted on to it). The new iso would likely work with old save file/folder though - I'm not sure of that though - I guess the woof-CE build team would have to look into that.

EDIT: Ah, my attempt to build from master didn't work. I'll look into it. Seems maybe that branch too old in that it didn't seem able to use huge kernels (EDIT: no, it is just done differently - see EDIT2 below). I can likely modify for next version to also be able to use master branch.

changing line 297 in makepup script from:
huge_kernel_mod
to

Code: Select all

[ -e support/huge_kernel.sh ] && huge_kernel_mod
seems to be enough to get it going if wanting to use woof-CE-master, though it does then ask extra questions of the user before build commences. I'll check if some or all of these interactions can also be automated - not too big of a deal since basically works with above change anyway. I'm waiting on master branch build to complete now, to see if that is also successful right to the end.

EDIT2: There seems to be code in master branch builddistro script that does allow master branch to also build with huge kernel - I'll look into that further since can probably incorporate that into makepup.

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#25 Post by wiak »

In terms of easily adding further extra packages I'll also be looking into using an optional chroot into the created rootfs-complete directory since might prove possible/straightforward (maybe not) to install further pets from there directly after the main build and then complete the build of the iso after that. chroot would only work for compatible host/target systems, but x86_64 host can chroot into x86 32bit targets, so could be useful technique. Will leave consideration of that till last (along with gui frontend considerations) however since lots of other things to test as a priority.

EDIT: also chroot already used in 01builddistro-Z though haven't studied the code in depth yet. Alternatively might need to automate the process of putting a pet into local-repositories in relation to DISTRO-PKGS-SPECS-distro-version and packages-puppy -<whichever_relevant>. But these are longer term issues for now.

wiak
Last edited by wiak on Tue 29 Aug 2017, 07:32, edited 1 time in total.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#26 Post by rufwoof »

Thanks wiak

Last time I tried to build from Master I didn't get very far. Had expected that to be the 'stable' version that might have been used to periodically update (rebuild) from ... but evidently wasn't the case ... at least not for me and my (extremely) limited knowledge of WoofCE.

Had it worked, woofce'ing (rebuilding) seemed like a nice way to periodically update a install to the latest versions available in the Ubuntu/wherever repositories. Again I don't know woofce, but it seems from a distant perspective that it isn't maintained that much now (little interest) other than periodic bursts of interest perhaps. I might have looked in the wrong place, but most of the commits I saw were months if not years old.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#27 Post by wiak »

rufwoof wrote: Had it worked, woofce'ing (rebuilding) seemed like a nice way to periodically update a install to the latest versions available in the Ubuntu/wherever repositories. Again I don't know woofce, but it seems from a distant perspective that it isn't maintained that much now (little interest) other than periodic bursts of interest perhaps. I might have looked in the wrong place, but most of the commits I saw were months if not years old.
It is still maintained, but mainly it seems by sole individual Puppy forum member jlst. I don't know how easy it is to get rights to make commits or maybe has to be done through a few individuals - for Puppy longterm survival, the bigger the woof-CE development team the better IMO.

woof-CE is a huge and amazing system when you consider it does build from scratch like a Puppy debootstrap (but without the huge Debian team). Despite rumours to the contrary, it seems to work very well indeed for the most part. My only gripe about it is that the code seems to have grown from small seeds in a less than modular fashion - no big deal, except some of the code sections (even if statements) are huge! Should be modularised - which is probably part of what jlst works on. Much easier to develop and maintain thereafter. I could have produced a working makepup script in a day or maybe two, but I actually spent over a week working on the structure for that very reason that it makes later development and maintenance work easier and shorter.

I was also concerned about how active or not woof-CE development is, since I think, especially from the point of view of original creativity, Puppy is an important distribution. So that's also indeed one of the reasons I decided to work on makepup - it does however rely on woof-CE being maintained up-to-date and without major flaws, though does allow quick hacks to build scripts should they be required to address a woof-CE script issue.

Perhaps it is possible to periodically update Puppy iso via woof build like this since woof itself is going upstream (to ubuntu and so on) for required packages.

Main thing for me is that Puppy is not in competition with Debian live and it doesn't matter if Debian live based systems can do everything Puppy can. There remains something precious about something original and nothing wrong with using stripped-down upstream packages from larger distributions; main thing is how the system is put together). Building DebianLive type systems is reasonably straightforward of course (since debootstrap does the core work), but the challenge/endeavour of Puppy is what to me makes it exciting. I wouldn't mean to take away any credit for the DebianDog type efforts, however - many great ideas/implementations coming from there.

wiak

jd7654
Posts: 296
Joined: Mon 06 Apr 2015, 16:10

#28 Post by jd7654 »

wiak wrote:...
woof-CE is a huge and amazing system when you consider it does build from scratch like a Puppy debootstrap (but without the huge Debian team). Despite rumours to the contrary, it seems to work very well indeed for the most part.
...
The problem with woof-CE is not the basic build. The first time I tried it, followed the instructions, it worked perfectly and created a working Puppy ISO just fine. That's not the problem. The problem comes when you try to customize things. Changing desktop, adding/removing packages and other customizations, that's where the trouble begins.

Just like Belham noted in his post:
http://murga-linux.com/puppy/viewtopic. ... 596#963596
Once you get past the basic woof-CE scripts, which your script seems to wrap up in one, then the hard part begins.

I think that's one of the reasons the DebLive script was so intriguing. While it did not create a complete Dobian Dog with all the bells and whistles like woof-CE creates a complete Puppy, its process of customizing things was pitifully easy. Change desktop, add apps, simple as adding the package names in a list.(compare that to DISTRO_PKG_SPECS)

I don't know if making woof-CE super easy is really the solution. Woof started as a Puppy builder, and not as a general Puppy construction set in and of itself. To really master it, having some knowledge of how Puppy works and how to build a Linux distro is required. Not sure if there is an easy way to shortcut that. But more power to you for trying and bring woof-CE forward to a new generation.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#29 Post by rufwoof »

My lack of understanding. Reading around some more I came across this that also links to http://blog.puppylinux.com/?viewDetailed=00025

So I think Master is a old no longer touched branch and primarily its testing (which is very active) that should be referenced/used to build from ???

In which case there looks to be no stable/release/snapshot or whatever you might call it branch (that I wrongly ???) assumed Master to be.

Which is fine assuming you're OK with tracking the latest updates and are OK with the standard build (choice of desktop, programs ...etc.). On the assumption you are then you could just rebuild periodically to have a snapshot of all of the latest changes/fixes/updates/security patches. A one line script to kick that off is great, fire it off, go to bed ... in the morning a new ISO sitting there ready for you. Best if you run more or less that standard version as-is and keep changes to a minimum (otherwise trying to merge in savefolder contents risks corruption/conflicts), which again is fine as Puppy makes it easy to setup things up in first-run-setup anyway. And just link in your docs, diary (osmo calendar/notes), browser bookmarks ... etc. to outside of Puppy space (HDD, which I tend to do anyway). A enhancement might be if the script could also cater for some of that ... such as UK locale, Europe/London timezone, en-GB keyboard, firewall on, numlock on ...etc. and perhaps saving/importing your default /etc/xorg.conf.

Or take the even easier route :) and just update your ISO whenever a new build is provided, for instance I've been tracking XenialPup64 http://distro.ibiblio.org/puppylinux/test/xenialpup/ and 7.0.8.5 is working really well (the only thing I've been adding to that is xterm as I like having a undecorated borderless terminal as part of the desktop and I use xterm for that).

Image

For that, once xterm is installed from PPM I add to ~/.jwm/jwmrc-personal

Code: Select all

<Group>
    <Name>xterm</Name>
    <Option>nolist</Option>
    <Option>noborder</Option>
    <Option>notitle</Option>
    <Option>sticky</Option>
    <Option>nopager</Option>
</Group>

<!-- had to add scaleheight 1.1 otherwise underscore character not seen -->
<StartupCommand>xterm -sh 1.1 +sb -uc -geometry 80x20+380+270</StartupCommand>
Those geometry sizes are specific to the 1280x800 resolution that I like to set in my xorg.conf (i.e. scaled down 16:9 ratio as I find the 1400x900 standard text display too small)

Code: Select all

Section "Monitor"
   Identifier   "DVI-I-2"
   VendorName   "Monitor Vendor"
   ModelName    "Monitor Model"
Modeline "1280x800_60.00"   83.50  1280 1352 1480 1680  800 803 809 831 -hsync +vsync
   Option "PreferredMode" "1280x800_60.00"
EndSection
(as per modeline determined by running cvt 1280 800)

along with xterm font settings in ~/.Xresources of

Code: Select all

Xterm.*font:                     10x20
XTerm*faceName: DejaVu Sans Mono
XTerm*faceSize: 10
EDITED to correct the 1st of the first two links (both originally linked to the same web page)
Last edited by rufwoof on Tue 29 Aug 2017, 02:06, edited 3 times in total.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#30 Post by wiak »

Ah, thankyou rufwoof, that issue 789 answered quite a few of the questions I had in my own head. It seems rationalise wasn't merged in the end into testing and clearly hasnt been deleted. I did start makepup off with testing branch as default but that proved risky since changes can occur rapidly in that branch and thus human error of course. We will see how matters upstream develop and I'll endeavour to keep up with that and try to add downstream enhanced additions via makepup. Main thing for me about it is just to provide a no questions asked one script build script with added options for more knowledgable wusers. Sorry, posting from Avery small-screen android just now and hard to see what I'm typing...

Wiak

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#31 Post by rufwoof »

A nice simplification IMO for ease of use would be some guidance as to a appropriate choice of kernel to go with the choice of installation.

I found the basic woof-ce build instructions easy to follow/do once they were highlighted, but did hit a barrier when asked which of 24 or so different kernels to use. A default ENTER choice that perhaps picked the LTS/default Ubuntu or whatever default kernel for the build would have been nice given that it takes quite a while to run through the build and I had chosen the wrong choice (having just opted for a higher kernel version number that wasn't really compatible).

Might be best to even just restrict to either the Ubuntu (xenial now that tahr is EOL) or Slacko (Debian or Slackware main trunks) core Puppy Choices. Nice and easy for new users to perhaps install via pre-built ISO initially, but later "compile" (build) their own updated versions (security updates/upgrades). And then perhaps only going on to native woof-ce for other choices as their understanding develops.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#32 Post by wiak »

rufwoof wrote:A nice simplification IMO for ease of use would be some guidance as to a appropriate choice of kernel to go with the choice of installation.
...
Might be best to even just restrict to either the Ubuntu (xenial now that tahr is EOL) or Slacko (Debian or Slackware main trunks) core Puppy Choices
Yes, that is a good tactic rufwoof. Main reason I haven't made any simplification like that thus far is that I don't myself yet know the best combinations to offer...

Hopefully, experienced pup builders (such as Billtoo) will feedback (in detail) other combinations that work really well and then I can use that info to provide the best default no commandline argument results (aside from choice of maybe Debian Stretch, Ubuntu Xenial or latest Slackware based). An option switch or switches for more advanced users would be used to turn on more complete choice control for the user and an eventual optional gui to make things easier still for some. Hopefully no-one thinks I've released makepup just as a quick way to build the default distro it provides - that's just an easy exemplar. Rather, I've always hoped for people to try building other pups with it and feed back what best works.

One of the good things about makepup routines, I hope/think, is that they can also be used to modify default woof script build actions (in fact that's what they do), which may come in handy at times for those who like hacking existing scripts.

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#33 Post by wiak »

Just a quick try to build a Debian Dpup Stretch iso with:

Code: Select all

./makepup -d 1 -r 1
where --distro 1 (-d1) produces Debian-based pup distribution
and --release 1 (-r1) produces Stretch release

Using makepup default woof-CE-rationalise branch.

I forgot to choose a kernel so just used the makepup default kernel number 24, which turned out to be:

Code: Select all

# uname -a
Linux puppypc6641 4.1.38 #1 SMP Thu Mar 9 23:23:15 +08 2017 i686 GNU/Linux
EDIT: No... It wasn't that I 'forgot to choose a kernel', it's because for stretch, _00build.conf forces choice of kernel to 4.1.38 via included line:

Code: Select all

KERNEL_TARBALL_URL=http://smokey01.com/ttuuxxx/WoofCe/kernel-4.1.38-stretch/huge-4.1.38-stretch.tar.bz2
It built fine and booted fine and I'm posting from it now using the distribution-provided NetSurf browser. Only issue was the desktop icon for the webbrowser didn't work so I had to start netsurf-gtk from the commandline. I think there are a few more issues, but that's a woof-CE Debian Stretch branch issue, not makepup problem really.

Screenshot attached.

wiak
Attachments
screenshot_makepup_stretch.png
Dpup Stretch build with makepup script
(71.06 KiB) Downloaded 741 times
Last edited by wiak on Wed 30 Aug 2017, 01:51, edited 1 time in total.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#34 Post by rufwoof »

Nice.

Was that the default desktop layout? ... as it looks different to the more usual layout/arrangement (tray that spans the entire width of the display, and icons staggered in the top left sloping left-downwards).

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#35 Post by wiak »

Hi rufwoof,

makepup just runs the official woof build scripts and auto-answers some choices so basically the build result is the default stretch woof-CE provides. Whether any change to the auto-answers would change anything I don't know off the top of my head but doubt. What could be changed is to add a few extra packages or firmware (if available); choice to be made from those indicated as 'no' in DISTRO-PKGS-SPECS-'distro-version', the names of the extra ones wanted should be manually appended to makepup_extra.conf (using normal txt editor) in the form:

Code: Select all

<package_name>="true"
That package adding could conveniently be achieved by temporarily forcing the makepup auto-build script to pause at appropriate place by starting the makepup build command with option -p. During the pause, DISTRO-PKGS-SPECS-distro-version could be viewed to see what packages are available to be added and then makepup_extra.conf edited appropriately followed by unpausing/continuing makepup script (by pressing 'y' Enter):

Code: Select all

./makepup -d 1 -r 1 -p
could also change kernel used by also adding to the above makepup commandline:

Code: Select all

-H <kernel_number>
wiak
Last edited by wiak on Tue 05 Sep 2017, 23:16, edited 1 time in total.

Post Reply