Auto-build a Puppy iso; single script with optional gui

[wiak]

Hi David and peebee,

I'm not on a Puppy at the moment. If I was, easiest is just to run makpup in gui mode with 'makepup -g' and the drop down will tell you which numbers to use for which distributions for those using commandline mode such as yourself david. However, I think it is:

Code: Select all

-t --target (TARGETARCH): 1 is arm; 2 is x86; 3 is x86_64

-d --distro (COMPATDISTRO): 1 is debian; 2 is devuan; 3 is slackware; 4 is trisquel; 5 is ubuntu

-r --release (COMPATVERSION): 1 is UbuntuPupBB32 or UbuntuTahr64; 2 is UbuntuTahr32 or UbuntuXenial64; 3 is UbuntuXenial32

In which case, for UbuntuPupBB32 you would want (if I've coded correctly? - which is looking doubtful...):

Code: Select all

-t 2 -d 5 -r 1

Or here are the code lines that I use in makepup to find the different combinations:

Code: Select all

    <text><label>$(gettext 'target architecture [-t]: ')</label></text>
	<combobox>
     <variable>TARGETARCH</variable>
     $(combobox_list 2:x86 1:arm 2:x86 3:x86_64)
    </combobox>

    <text><label>$(gettext 'distro base [-d]: ')</label></text>
    <combobox>  
     <variable>COMPATDISTRO</variable>
     $(combobox_list 3:slackware 1:debian 2:devuan 3:slackware 4:trisquel 5:ubuntu)
    </combobox>

    <text><label>$(gettext 'release version [-r]: ')</label></text>
    <combobox>  
     <variable>COMPATVERSION</variable>
     $(combobox_list 3:Slackware14.2 1:DebianStretch 1:DevuanAscii 1:Slackware14.0 1:UbuntuPupBB32 1:UbuntuTahr64 1:TrisquelBelenos 2:Slackware14.1 2:UbuntuTahr32 2:UbuntuXenial64 3:Slackware14.2 3:UbuntuXenial32)
    </combobox>

However, when I upgraded makepup to hopefully find UbuntuPupBB32, I didn't really test it because was in middle of other work, but relied on feedback to tell me if I got it wrong. According to what peebee has written in post above, I seem to have indeed got the code in makepup gui wrong, but my system has run out of space so I have no Puppy on here to test, so please let me know if I have to change the combinations in makepup code and I'll fix it...

EDIT: Just had a look at woof-CE on github and I note that trisquel seems to have been removed, which would change the number ordering. Alas, whoever makes these numeric changes didn't inform me and that ordering is of course required by makepup. Please let me know what current numberings are and I'll alter makepup accordingly (looks like ubuntu in now --distro 4, or maybe trisquel is still there and I've just not noticed it - been a while since I checked).

EDIT: Looks like -t 2 -d 4 -r 2 might indeed be correct and I'll have to modify makepup accordingly. If so, I'll install a pup to a usb in the next day or less and do a quick manual woof-CE puppy build to check the current numeric values needed and modify makepup accordingly - sorry about that...

wiak

[davids45]

G'day wiak & peebee,

Thanks for your quick and informative replies about my number choices in makepup's config not giving me a BionicPup.

I'll try 2-4-2 and if that's no good, maybe 2-4-1.

Fairly quickly if I watch what's happening on my monitor, I can see the basic Pup being built - if I don't like it, I can stop makepup. Last night I tried 2-5-4 and could see it was still a Xenial Pup being made, so just killed makepup on the terminal, shut down the computer, and went to bed.

Once I get the right Bionic settings, I can try removing default packages I don't want and adding in my preferred ones in one form or another, plus add config files and the like (as .pets) for my desktop.

Then I can try switching kernels to maybe improve boot speed and general Puppy operation.
Incomplete pinboard display during booting is my main new-Puppy bugbear at present, fixed with clicking a pinboard icon linked to a script to restart-X. Maybe I'm loading too many sfs files (with pinboard icons) and have too many partitions (lately, these are very slow to display across the pinboard).

Anyway, makepup should let me try options to improve my particular/peculiar situation (if possible).

Thanks both again.

David S.

[wiak]

Hello David,

Yes, the loss of trisquel changed all the numbers so I'll have to upload a different makepup version. In the meantime, post 2 of this thread (at top) gives some examples of current numeric values that should sort your problems out. Meanwhile, I'll fix the numbers in next release for makepup.

Current numeric values examples in post 2 of this thread here:

http://murga-linux.com/puppy/viewtopic. ... 543#965543

So, currently, for 32bit upupbb, you do indeed need -t 2 -d 4 -r 2

wiak

[wiak]

makepup version 0.1.6 uploaded.

To first post of this thread, here:

http://murga-linux.com/puppy/viewtopic. ... 541#965541

Following Trisquel selection being removed in woof-CE, this new makepup version hopefully fixes numeric values for distribution selection, but needs testing, thanks.

uPupBB32 is currently the default build (i.e. -t 2 -d 4 -r 2)

But please note the following post from PeeBee regarding kernels for uPupBB32:

http://murga-linux.com/puppy/viewtopic. ... 107#994107

EDIT: Note that you do get one popup window during uPupBB32 build (and maybe any Ubuntu?); I can't remember why sorry or if you can just ignore it (please let me know), though I think someone drew attention to this earlier in the thread somewhere... I'd have to study the woof-CE scripts again to remove that popup and don't have time at the moment, sorry. EDIT2: I think now that the popup just vanishes of its own accord if ignored.

EDIT2: I wouldn't recommend building a pup on a usb stick via usb2 interface - that takes forever (very slow), but fine on a hard drive partition in my experience.

wiak

[davids45]

G'day wiak,

Thanks for taking the time to update makepup to 0.1.6. I'll give it a go or two later today.

Regarding the kernel to be used, is it still true to your recollection that putting the 'huge-kernel' tar.bz desired into the /local-repositories/kernel2add directory will override any other kernel in the config file (or will if I delete any kernel number in the config file line about the kernel to use)?

David S.

[wiak]

Regarding the kernel to be used, is it still true to your recollection that putting the 'huge-kernel' tar.bz desired into the /local-repositories/kernel2add directory will override any other kernel in the config file (or will if I delete any kernel number in the config file line about the kernel to use)?

David S.

Yes, am far as I remember that should override other kernels that appear in the hugekernel list (what you can see in dropdown list when running makepup in gui mode). There is a config file, _00build.conf, where a distro-builder COULD specifically specify a particular kernel that MUST be used: for example debian stretch 32bit build has specified in its _00build.conf:

KERNEL_TARBALL_URL=http://smokey01.com/radky/Woof/kernel-4 ... ch.tar.bz2

and to use a different kernel the builder would first need to comment out that line. But generally distribution creators on woof-CE don't seem to have forced kernels in that manner. For example, in the case of upupbb that KERNEL_TARBALL_URL line is already commented out in its _00build.conf so just drag/dropping new huge kernel into that kernel2add directory provided by makepup should work fine I think.

wiak

[wiak]

makepup 0.1.7 released. Download from first post of this thread.

Hopefully fixes all the numbering to match current woof-CE-testing and woof-CE-master available distros. This needs tested though, and feedback if I'm to fix any errors. I only have rural broadband so proved too slow for me to complete a build as yet - the Pups are getting bigger so taking longer and a lot more bandwidth to make test builds. (Though second time I try later tonight should be fine since makepup allows -k option to keep already downloaded packages).

DPupBuster isn't on woof-CE github yet, so not included in makepup lists for now.

Simple usage is to create empty build directory (on filesystem that has lots of space on it...); put makepup script into that and make it executable with chmod +x; and then run it in gui mode with makepup -g; then uncheck the -k keep previous downloads advanced option, and press "Build your pup!". It takes ages... However, after first build you can rebuild another one without having to download all the distro packages again (just specify the -k keep advanced option when running makepup -g this other time...). So handy for experimenting with different builds. I do not recommend building in usb2 mounted flash drive - probably far too slow for the work woof-CE scripts need to do.

BionicPup64 is currently the default build (i.e. -t 3 -d 2 -r 1)

Now that scottman's package manager program is available (and on woof-CE I see), I believe I could now add a lot more power to makepup operation (future version) in terms of woof building pups very flexibly (assuming the pkg program resolves dependencies). Actually, I could already enhance makepup to do that without needing any upstream mods to woof-CE build scripts since makepup re-writes these scripts for its own purposes on the fly anyway (which is how makepup's pets2add and pkgs2add facility works; however would work much better with dependency resolving cmdline package manager such as scottman's - I would just need to add the facility to use that into makepup, though I am unsure if I want to do any more developing. thinking about it.).

Please let me know if you get round to testing this new makepup since I am unlikely to fix any issues otherwise whilst my own testing is so limited by broadband download speed issues.

wiak

[foxpup]

however would work much better with dependency resolving cmdline package manager such as scottman's

That's exactly one of the goals sc0ttman had in mind with pkg!
I love it when a plan comes together .

[rerwin]

wiak,
Thank you for continuing to maintain makepup.

Back in January, I experimented with makepup 0.1.6 and encountered a few difficulties, some of which I coded fixes for. You might find them useful additions to the official makepup.

1. I was unable to access pet packages that were xz compressed, so replaced:

tar xzf "${rep}/$dotpet" -C "$rep" --no-anchored pet.specs 2>/dev/null

with:

case $(file -b "${rep}/$dotpet") in
gz*|GZ*) tar xzf "${rep}/$dotpet" -C "$rep" --no-anchored pet.specs 2>/dev/null ;;
xz*|XZ*) tar xJf "${rep}/$dotpet" -C "$rep" --no-anchored pet.specs 2>/dev/null ;;
esac


2. I was uncertain as to when I needed to have pets resident in the local repo, so changed:

Enter y when ready to continue, any other key to quit: '

to:

All user-provided pet packages need to be in /local_repositories/pets2add before proceeding.
All user-provided distro-specific (.deb) packages need to be in /local_repositories/pkgs2add before proceeding.
Enter y when ready to continue, any other key to quit: '


3. I saw some directory names that contained double slashes, so removed the final "/" from:

local rep="../local-repositories/pets2add/"

I did not see any bad effects from removing the slash. I assume the same change should be made to:

local rep="../local-repositories/pkgs2add/"


I am hoping to work with 0.1.7 soon.

Richard

[wiak]

thank you rerwin/Richard. I'll implement your fixes in next version. I have come across another issue though not an 'error' per se. The hugekernel list is simply what the relevant ibiblio webpage provides, but that includes the sha256 and md5 files so it is easy to accidentally select a checksum file rather than the desired hugekernel itself. (As just happened to me in testing...). For now I'm just going to change the default kernel number from 24 to something else since 24 pointed to an sha256 file...

wiak

[wiak]

I've taken of makepup ver 0.1.7 for now, to modify hugekernel default pointed to. Since I'm going to upload changed version as 0.1.8 I might as well implement rerwin's suggested fixes for users to test.

Will modify and upload ver 0.1.8 as soon as I can, but may need to sleep first. If you have already downloaded ver 0.1.7 I would still appreciate any tests done using that, but be very careful to make sure the hugekernel selected is the actual hugekernel file rather than its md5 or sha256 check file.

EDIT: Hmmm... After, what I thought would be a fix, I've still got a problem with that sha256 file being thought of as the kernel... I'm working on it. Now trying a straight woof-CE build to see what the difference is.

wiak

[wiak]

While I'm at it I'll modify the hugekernel list processing to sed out sha256 entries (I guess that checksum wasn't previously being used).

wiak

[wiak]

however would work much better with dependency resolving cmdline package manager such as scottman's

That's exactly one of the goals sc0ttman had in mind with pkg!
I love it when a plan comes together .

Yes, indeed. I actually commented to sc0ttman when he first announced early creation of pkg how that was exactly what I needed to expand makepup (or even better to directly enhance github woof-CE scripts themselves).

Fredx181's mklive-stretch script actually uses similar idea in that after using debian debootstrap script(s) for the initial build, he then uses apt/dpkg commandline dependency resolving package manager to fetch all the optional bits wished for in the final iso build. That ability to separate base from apt/dpkg fetch part is what allows the amazing flexibility in the overall iso build. It is that dependency resolution capability that is missing in current woof-CE scripts that pkg could provide and/or makepup use in its pets2add and pkgs2add functionality, which is already an enhancement to what woof-CE itself offers; however, currently that makepup-provided additional functionality cannot resolve dependencies... hence my desire and often remarked enthusiasm for continuing development of sc0ttman's pkg program!!!

So basically woof-CE could do with being able to provide that similar to mklive-stretch capability, and it needs pkg (or similar for that). Anyone who thinks the Dogs do not provide essential ideas for the future of Puppy Linux woof-CE development are somewhat 'blind' or sometimes technically lacking in understanding. Those of us who do dev work for both systems understand very well the importance of cross-pollination efforts for both systems. It is only correct and fair, therefore, that all such systems, Pups and Dogs, are equally respected and promoted on this forum.

wiak

[wiak]

I won't for now upload my next makepup version until issue checked/fixed(?) at woof-CE repo since I think that is where the problem is - I've raised issue on woof-CE github accordingly. Many thanks in advance to the woof-CE team for looking into this for me.

Easiest 'fix' being, I believe, for ibiblio repo to use md5.txt file where it currently has a sha256 file. Alternatively, I believe the woof-CE support/hugekernel.sh script would changed to handle sha256 files rather than just the md5 checksums processing it does just now.

wiak

[wiak]

makepup 0.1.8 now released. Download from first post of this thread.

Includes rerwins fix for dotpet compressed with xz (not tested by me but looks fine). mavrothal has confirmed the issue I identified to woof-CE github team and seems they will simply rename the errant files at huge kernel repo, as I suggested, so nothing needs changed in makepup concerned with that.

I've completed one BionicPup64 (default build) with makepup and using pets2add included my dotpets for wex, weav, and scrox at build time. iso was produced successfully and these additions were found in it, so that one worked!

This version hopefully fixes all the numbering to match current woof-CE-testing and woof-CE-master available distros. This needs tested though, and feedback if I'm to fix any errors. I only have rural broadband so proved too slow for me to complete a build as yet - the Pups are getting bigger so taking longer and a lot more bandwidth to make test builds. (Though second time I try later tonight should be fine since makepup allows -k option to keep already downloaded packages).

DPupBuster isn't on woof-CE github yet, so not included in makepup lists for now.

Simple usage is to create empty build directory (on filesystem that has lots of space on it...); put makepup script into that and make it executable with chmod +x; and then run it in gui mode with makepup -g; then uncheck the -k keep previous downloads advanced option, and press "Build your pup!". It takes ages... However, after first build you can rebuild another one without having to download all the distro packages again (just specify the -k keep advanced option when running makepup -g this other time...). If you are adding extra pets by dropping them into pets2add folder, remember to check Advanced option -a box prior to pressing Build.... So handy for experimenting with different builds. I do not recommend building in usb2 mounted flash drive - probably far too slow for the work woof-CE scripts need to do.

BionicPup64 is currently the default build (i.e. -t 3 -d 2 -r 1)

Please let me know if you get round to testing this new makepup since I am unlikely to fix any issues otherwise whilst my own testing is so limited by broadband download speed issues.

wiak

[rerwin]

wiak,
I am experimenting with makepup-0.1.8 and several added pet packages. The noarch pets (e.g., ultrasns-20190310) produce correct DISTRO_SPECS entries (yes|ultrasns||exe|pet).

However, architecture-specific pets (e.g., minibase-net-20190301-x86_64.pet) produce incorrect entries (yes|minibase-net-20190301||exe|pet).

Changing makepup line 105 from:

tac DISTRO_PKGS_SPECS-* | sed '0,/'"'"'/ s//'"'"'\nyes|'"${dotpetname%-*}"'||exe|pet/' | tac >/tmp/makepup/DISTRO_PKGS_SPECS # needs GNU sed

to:

tac DISTRO_PKGS_SPECS-* | sed '0,/'"'"'/ s//'"'"'\nyes|'"${dotpetname%%-[0-9]*}"'||exe|pet/' | tac >/tmp/makepup/DISTRO_PKGS_SPECS # needs GNU sed

produces the correct entry (yes|minibase-net||exe|pet). Do you think that change is sufficient?


However, the architecture-specific pets appear to be destined for puppy-noarch-official, when they should go to puppy-common64-official (common32-official), or maybe the distro-official. Makepup does not appear to address this type of pet. What to do about that?

(As you can see, I am experimenting with a puppy adaptation of Barry's UltraSNS.)

[wiak]

Changing makepup line 105 from:

tac DISTRO_PKGS_SPECS-* | sed '0,/'"'"'/ s//'"'"'\nyes|'"${dotpetname%-*}"'||exe|pet/' | tac >/tmp/makepup/DISTRO_PKGS_SPECS # needs GNU sed

to:

tac DISTRO_PKGS_SPECS-* | sed '0,/'"'"'/ s//'"'"'\nyes|'"${dotpetname%%-[0-9]*}"'||exe|pet/' | tac >/tmp/makepup/DISTRO_PKGS_SPECS # needs GNU sed

produces the correct entry (yes|minibase-net||exe|pet). Do you think that change is sufficient?

However, the architecture-specific pets appear to be destined for puppy-noarch-official, when they should go to puppy-common64-official (common32-official), or maybe the distro-official. Makepup does not appear to address this type of pet. What to do about that?

I'm happy you are experimenting with makepup rerwin. It is such a long time since I wrote these routines that I am utterly rusty about them now. I don't think many people tested this previously, or at least I had no feedback, and I only tested quickly with a few simple dotpets of my own. At a glance, your mod looks good, but only time will tell. I will test it briefly and upload new version thereafter in a few days time. As far as the list the entries appear in, yes, at the time I made this makepup addition I was mainly just testing a proof of concept and, imprecisely, just using puppy-noarch-official. But when you say makepup does not appear to address this type of pet, what exactly do you mean - if you drop such a pet into pkgs2add folder (with the line 105 fix you mention, do they not get included in the build anyway? - to be honest, not having looked at the code for so long, I can't remember what my code does... well... I 'roughly' still know). I may say, I am perfectly happy for you to make major developments to makepup, and take it over if you wish - I'm trying to slow down my involvement on Puppy more generally nowadays. Since you have an interest in using makepup, perhaps you would be interested in that? I'm sure you could improve it greatly.

wiak

[rerwin]

I mean that it does not put the x86_64 pets in the right repo. But it does seem to include them in the build. So, I probably will need to move some items among the Package...official files later, if I update them in woof-CE (wherever they are). My hopefully successful build is running now and I will work with it tomorrow -- bedtime, now.

My additions to the build are some noarch and arch-specific pets and replacing the wvdial pet with the deb from ubuntu, learning as I go.

[wiak]

I mean that it does not put the x86_64 pets in the right repo. But it does seem to include them in the build. So, I probably will need to move some items among the Package...official files later, if I update them in woof-CE (wherever they are). My hopefully successful build is running now and I will work with it tomorrow -- bedtime, now.

Yes, I guess makepup should be updated with similar blocks of code to select, or at least auto-selecting, which arch-specific distro list the addon info gets sent to. Clearly it is a nuisance having to make the later item moves around official files later. Just using 'noarch' was admittedly a quick fudge on my part!

wiak

[rerwin]

Report on my makepup build of XenialPup-7.5:

The build went almost smoothly overnight. But the memory tray icon showed that there was 0 free memory in the save partition, even though Partview showed about a gigbyte free in the partition. (I use a save directory for the build.) There were no obvious error messages, although 2 pets were claimed to be not found -- except that at least one of them ended up in the ISO. (That was wvdial, where I supplied the deb file in add2pkgs. I suspect it is not actually in the woofCE copy of the ubuntu repos.)

The resultant puppy actually booted up normally but the background is black and no icons and few tray icons appear. I see these messages in xerrs.log:

• sed: can't read /root/Choices/ROX-Filer/PuppyPin: No such file or directory
  grep: /root/Choices/ROX-Filer/PuppyPin: No such file or directory
  /sbin/clean_desk_icons: line 9: can't create /root/Choices/ROX-Filer/PuppyPin: nonexistent directory
  grep: /root/Choices/ROX-Filer/PuppyPin: No such file or directory
  cat: /root/Choices/ROX-Filer/PuppyPin: No such file or directory
  cp: cannot create regular file '/root/Choices/ROX-Filer/PuppyPin': No such file or directory
  /root/.xinitrc: line 132: roxfiler: command not found
  /usr/bin/jwm

ROX-Filer is where is should be but there is no /root/Choices/ROX-Filer directory, so no PuppyPin file. The above messages may have been generated when I attempted to run ROX-Filer from the Filesystem menu.

I am not sure what to make of it all, as I have not mastered that desktop management stuff. I will keep digging, but want to give you some feedback, such as it is.


Some other disconcerting messages in xerrs.log:

• (process:4120): GLib-GObject-WARNING **: Two different plugins tried to register 'GProxyResolverGnome'.
  
  (process:4120): GLib-GObject-CRITICAL **: g_type_add_interface_dynamic: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed
  
  (process:4120): GLib-GIO-WARNING **: Tried to register an extension of the type (null) to extension point gio-proxy-resolver. Expected type is GProxyResolver.
  
  /usr/local/bin/rox: line 2: exec: roxfiler: not found