Alternative way to build Ubuntu / Debian Puppy [RETIRED]

[01micko]

Thanks Iguleder - using your DOTconfig with 3.12.21 with aufs added. Just raw, as in I haven't checked if new modules are available. Compiling now.

I'll add James' firmware to a kernel-modules.sfs and see if the thing boots - more soon.

EDIT: BTW 3.12 is LTS too and what slacko64 was using before the 3.13 experimental release.

EDIT: failed to boot, initramfs problem, anyway rebuilding using fatdog-630-rc2 config ..which is what i had on hand ..should "just work"

EDIT: FD config booted but no keyboard at console for me. I honestly don't know if it worked in fatdog itself at console on this machine so I'll have to check. (LATER: checked .. yes it did work in FD-630-rc2 k3.12.6)

NEXT Morning
EDIT: depmod call in /etc/rc.d/rc.sysinit fails so that's why kernel modules didn't load. I ran depmod on the kernel modules before squashing and now it works . I should be able to mod kernel kit appropriately to produce a fatdog style kernel.

Code: Select all

Linux puppypc29976 3.12.21 #1 SMP Sun Jun 8 07:25:04 EST 2014 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/DISTRO_SPECS
DISTRO_NAME='ubuntu Puppy'
DISTRO_VERSION=7.0
DISTRO_BINARY_COMPAT='ubuntu'
DISTRO_FILE_PREFIX='ubuntu'
DISTRO_COMPAT_VERSION='ubuntu'
DISTRO_XORG_AUTO='yes'
DISTRO_TARGETARCH='x86'
DISTRO_DB_SUBNAME='ubuntu'
DISTRO_PUPPYSFS=puppy.sfs
DISTRO_ZDRVSFS=kernel-modules.sfs

Ignore the $DISTRO_TARGETARCH.. uname certainly wants us to know that this is x86_64, 3 times

[01micko]

I just did a fresh install of slacko-5.7 to test out kernel-kit modifications on 32 bit and the Ubuntu install using tahr bins. I have just pulled james' latest so it will have the mods he put in (so far I haven't updated since initial pull).

If all goes well I'll post kernel kit here for testing and if that goes well I'll push it to woof-CE/testing. Kernel is compiling now. (again 3.12.21). Kernel kit will have both configs. BTW, it's 4G not PAE so I can test on my old centrino lap top. That can be easy changed in the config for those who like PAE, and beauty of the FatDog kernel method is simplicity in switching kernels. Just replace vmlinuz and kernel-modules.sfs and reboot.

Mavrothal.. it might actually be time soon for a new git repo.. woof-NG

[stemsee]

This new kernel, initrd and kernel-modules.sfs combination, has to be the most significant breakthrough in Puppy Development for ages. The time I have wasted fiddling around with DISTRO-SPECS in main.sfs and initrd and swapping modules around and separating modules for initrd, and/or compressing/decompressing *.ko.gz mods, and all the other stuff was utterly unnecessary! And this method will, hopefully, prove it!

Kudos to you. Anything I can do let me know!

Edit: Any chance of ensuring all puppys build with the ability to 'xinput create-master second' , only Lighthouse64 has that ability and after research and experimentation I can shed no light no the issue. It is the one thing which gives superiority to Debian based systems in my opinion.

[01micko]

Ok, 32 bit kernel is a success and posting from it now, just added dhcpcd to the default, hacked build-iso.sh to grab the kernel from my personal repo and added firefox and we are away.

Code: Select all

# uname -a
Linux puppypc27288 3.12.21 #1 SMP Sun Jun 8 15:24:18 EST 2014 i686 athlon i686 GNU/Linux

More later.. dinner time!

[01micko]

Time for some code

Diff to build iso (grabs kernel)

Code: Select all

--- build-iso.sh.orig	2014-06-03 20:54:06.660580810 +1000
+++ build-iso.sh	2014-06-08 18:02:47.560002531 +1000
@@ -10,6 +10,7 @@ ISO_ROOT=${ISO_ROOT:-$OUTPUT_DIR/iso-roo
 PUPPY_SFS=${PUPPY_SFS:-puppy.sfs}
 KERNEL_VERSION=${KERNEL_VERSION:-3.12.9}
 PARENT_DISTRO=${PARENT_DISTRO:-ubuntu} # or debian
+KERNEL_URL=${KERNEL_URL:-http://distro.ibiblio.org/fatdog/kernels/700}
 
 WOOF_ISO_ROOT=${WOOF_ISO_ROOT:-boot}
 WOOF_INITRD=${WOOF_INITRD:-boot/initrd-tree0}
@@ -29,7 +30,7 @@ install_boot_files() {
 install_kernel() {
 	for p in vmlinuz kernel-modules.sfs; do
 		! [ -e $ISO_ROOT/$p ] &&
-		wget -P $ISO_ROOT -c http://distro.ibiblio.org/fatdog/kernels/700/$p-$KERNEL_VERSION
+		wget -P $ISO_ROOT -c $KERNEL_URL/$p-$KERNEL_VERSION
 		mv $ISO_ROOT/$p-$KERNEL_VERSION $ISO_ROOT/$p
 	done
 }
@@ -58,7 +59,7 @@ make_iso() {
 	-volid "Puppy-Linux" \
 	-iso-level 4 -D -R  \
 	-b isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table $ISO_ROOT/
-	isohybrid -o 64 "$OUTPUT_DIR/$OUTPUT_ISO"
+	isohybrid "$OUTPUT_DIR/$OUTPUT_ISO" #-o 64
 }
 
 ### main
# 

NB: The third hunk of the diff will not be necessary for FatDog (isohybrid part)

Apply that and run with KERNEL_URL as an arg (will work as I have uploaded that kernel)

Code: Select all

KERNEL_VERSION=3.12.21-slacko32FD4G KERNEL_URL=http://01micko.com/packages/ ./build-iso.sh

I'll attach kernel-kit adjusted to build a fatdog64 (or 32) bit kernel.

[stemsee]

I'll attach kernel-kit adjusted to build a fatdog64 (or 32) bit kernel.

Question on kernel-kit-ng

''# DO NOT set it to 0 (zero) >> cooked machine
JOBS=-j6''

Having -J0 results in cooked cpu, dual core cpu has 4 threads (-j4) kernel-kitt is set default to ''JOBS=-j6'' will that not cook a machine also unless it has 3 or more cores?

Also, ../dist/sources/patches directory missing/not created only a patches file, hence no patches to apply ... copied manually. Maybe that is how it is supposed to be ... what do I know??

Kernel compiled using new kit = 3.14.5-32FD-pae and sfs i don't have the new initrd which I imagine needs an amended init pointing to the modules.sfs so I haven't tested it. Also the DOTconfig has some significant changes, for example deselected RTC, and included additional fs, i486. Also, added firmware. The kernel gets dumped in dist/packages/ outside of the linux_kernel-*/boot directory.

kernel
https://drive.google.com/file/d/0B4GhZV ... edit?usp=1
modules.sfs
https://drive.google.com/file/d/0B4GhZV ... edit?usp=1

[mavrothal]

I'll attach kernel-kit adjusted to build a fatdog64 (or 32) bit kernel.

Looks good.
added dm_crypt in vmlinuz just in case
Aufs mm/fremap.c patching failed in 3.12
Also in line 396 the sfs sufix is in the middle
Another little issue is that will not empty or stop and warn idf the output directories are not empty so if you build 2 karnels (different conf or $FD) may end up in a mix up.

[01micko]

@stemsee

''# DO NOT set it to 0 (zero) >> cooked machine
JOBS=-j6''

Having -J0 results in cooked cpu, dual core cpu has 4 threads (-j4) kernel-kitt is set default to ''JOBS=-j6'' will that not cook a machine also unless it has 3 or more cores?

Well, I don't think jobs are the same as threads.. I should have commented it however I have been using -j6 forever on my athlon X2. Cooked machine is a possibility but more likely is the thermal cut out protection.

Also, ../dist/sources/patches directory missing/not created only a patches file, hence no patches to apply ... copied manually. Maybe that is how it is supposed to be ... what do I know??

There is intentionally only 1 patch in patches. Others are generated in the script and archived in dist/ for future reference.

Kernel compiled using new kit = 3.14.5-32FD-pae and sfs i don't have the new initrd which I imagine needs an amended init pointing to the modules.sfs so I haven't tested it. Also the DOTconfig has some significant changes, for example deselected RTC, and included additional fs, i486. Also, added firmware. The kernel gets dumped in dist/packages/ outside of the linux_kernel-*/boot directory.

There are 2 ways you can test a kernel resulting from the FD build.

• 1. build a distro using jamesbond's scripts
  2. build a 64 bit kernel and replace the FatDog64 kernel

I hope these answers help

-

@mavrothal

Aufs mm/fremap.c patching failed in 3.12

Yes I am aware. This needs to be reported to Junjiro and that's why I changed from "exit 1" to a pause and inspect routine.

Also in line 396 the sfs sufix is in the middle

Intentional to be compatible with jamesbond's build-iso.sh. As stated up the page, the diff needs to be applied so you can use the KERNEL_URL as a param. The suffix gets stripped off in the script; vmlinuz gets the same treatment.

Another little issue is that will not empty or stop and warn idf the output directories are not empty so if you build 2 karnels (different conf or $FD) may end up in a mix up.

Yes, I suppose I should add a "clean" param to the script and a note in README. I don't want to automate that, so much hand holding is ok but it's the kernel we are talking about here and if a user doesn't know what they are doing then they shouldn't be using the scripts. It doesn't check for space either

Thanks for reports.

[stemsee]

I built a debian with jamesbond's scripts 'build-deb'. However I realise that i misunderstood kernel-kit-ng... just have to recompile kernel and then insert for ./3builddistro

So to clarify. The kernel does not need to be in the ../dist/linux_kernel-*/boot directory nor does that directory need to dir2tgz-ed and tgz2pet -ed and added to local-repositories for inclusion in the build ?

[01micko]

James' script doesn't touch 3builddistro. I believe he only used the woof-CE framework so that the generic puppy scripts (including the initrd) could be imported to the build. After deb-build.sh run build-iso.sh. This pulls in the kernel from the fatdog ibiblio repo. My diff just adds an option to pull in a kernel from elsewhere and I uploaded mine to my repo to test.

To use your kernel, just comment the install_kernel line (near the bottom), add a read command directly under it, which effectively pauses the script so that you can insert your kernel (into iso/iso-root/) , continue the script and an iso will be built with your kernel. You could be more elegant than my hacky method if you want to spend the time to add the code pulling in a kernel from a local directory.

[stemsee]

No Wonder! duh!

[01micko]

Just make sure you rename your kernel image to vmlinuz and the module sfs to kernel-modules.sfs

[mavrothal]

Just make sure you rename your kernel image to vmlinuz and the module sfs to kernel-modules.sfs

Actually (James) with all different kernels per puppy and multiple puppies, using the generic vmlinuz and modules.sfs can be a problem. Symlinks would be a solution but we install in (V)FAT too...
Grub and the likes could use the full names but the init (and DISTRO_SPECS) should also be modify to recognize the specific kernel or put the kernel in the initrd (as in FD_ARM/XOpup). But then changing kernels would not be so easy. Maybe it shouldn't as a "random" kernel can really break your system.

BTW Mick, since you were posting while I was editing, I do not know if you noticed the comment about directories being empty, above.
BTW2 you do not need the git directories to be empty (or date stamped). Git reset/clean/{fetch,pull} should do it.
BTW3 I assume "NG" means next generation, however this is not nesseceraly the most common use of the initials...

[stemsee]

Edit: Booted normally into trusty tahr i.e. straight to xorgwizard = keyboard select then video select then straight to desktop!

Now posting from EmSee-2nd-Edition using new kernel, new initrd and kernel-modules.sfs only had to rename main sfs to puppy.sfs. Previous kernel posted here 3.14.5-pae works, but newer one is better, same version recompiled.

I think copying my previous main.sfs files into savefiles might be the easiest way to load them onto the puppy.sfs from the build. in order to build it up.

That was a truly painless process!!

Edit: Sharing my newest kernel 3.14.5-pae and kernel-modules.sfs and initrd in tar.gz folder for general usage. Kernel is compiled with timer @1000hz not 300hz as in 01micko's 3.12.21, also i486 not pentium pro, and extra functionality that might be useful to someoen, somewhere, sometime - maybe!

https://drive.google.com/file/d/0B4GhZV ... edit?usp=1

have a nice day!


EDIT: I noticed that loading my own base.sfs results in the boot directory not being mounted on /mnt/home but /initrd/mnt/pup_ro2 I didn't check if that was the case when booting the trusty puppy.sfs from the build.

[jamesbond]

Glad to hear the success stories.

Good work Mick!!
I've merged your changes build-iso.sh.

@stemsee, what do you mean by "xinput create-master second"?

Well, I don't think jobs are the same as threads.. I should have commented it however I have been using -j6 forever on my athlon X2. Cooked machine is a possibility but more likely is the thermal cut out protection.

No, no cooked machine unless your machine's thermal protection is faulty.
In general, the recommended number is 1.5x your CPU cores.
If you have too many jobs, then the overall result becomes *slower*, not faster.
If you really want to compile in a flash, and you have a few machines to spare, then learn to use distcc

Actually (James) with all different kernels per puppy and multiple puppies, using the generic vmlinuz and modules.sfs can be a problem. Symlinks would be a solution but we install in (V)FAT too...

Agreed. Just to note that build-iso.sh is a quick and dirty script
It is there just to show that once you have the SFS, you can easily create an ISO.
Please feel free to hack build-iso.sh to come with the proper naming scheme kernel and module.sfs, pointing DISTRO_PUPPYSFS and DISTRO_ZDRVSFS to the right name.

put the kernel in the initrd (as in FD_ARM/XOpup

I think you mean "put modules.sfs" in initrd. Yes, this is an option, this is what I actually do for Fatdog64/FatdogArm/XOpup. But note: putting it in initrd means it will stay in RAM. This may or may not be a good thing:
a) Fatdog64's module.sfs is around 40MB, and this 40MB will be kept around in memory (it *will not* show in "free", but it's memory that programs can't use nonetheless). For Fatdog64, this isn't a problem because it runs in a machine with 1GB memory. A comfortable 64-bit machine will have 2GB or more. What is 40MB for a friend
b) FatdogArm's modules.sfs is very small (around 8MB), and thus can stay there, as reasoned above.
Now, I didn't check the size of typical Puppy kernel's modules.sfs; and whether it is acceptable to hold *that* in RAM while Puppy is running.

One thing I didn't check when I put the modules.sfs into ZDRV - does ZDRV gets copied to RAM too at boot time? If not, that means when you use CD to boot it, the CD can't be taken out.

@Mav - NG does mean "Next Generation" - as you can see from "samba-tng", "util-linux-ng", and many other examples

Mick I saw you removed "-o 64" from isohybrid, is that because 32-bit kernels don't boot with that?

[stemsee]

xinput installed on puppy results in # xinput list commmand showing one mouse0 and keyboard0, the same command on L64 and all other linux distros results in a list of input devices as seen in hardinfo - input . a 'Master' is the head to which are attached the first keyboard and mouse and all other input devices. By creating a second master xinput allows you to attach to it a second keyboard and mouse which gives another pointer that acts indepedently of the first as in multi-touch. You can of course create several masters with several pointers and keyboards etc. It may be possible still to 'map' a master to a display/screen/desktop. Which is useful for me and my family to share one pc with extra displays keyboard and mice = pseudo multi-seat without nested servers = accelerated graphics and much better performance. Not for most people's usage I know.

Edit: Here is the patch for mapping input device to screen.

http://lists.x.org/archives/xorg-devel/ ... 23898.html

[mavrothal]

One thing I didn't check when I put the modules.sfs into ZDRV - does ZDRV gets copied to RAM too at boot time? If not, that means when you use CD to boot it, the CD can't be taken out.

If there enough RAM, yes

NG does mean "Next Generation" - as you can see from "samba-tng", "util-linux-ng", and many other examples

Depends where you look

[01micko]

--if suffix is -nfg then you absolutely know it's no good

james, as for the the -o 64 option, I think it's a case of old isohybrid in slacko, nothing more.

PS @jamesbond. re kernel-kit. Do you have a script for getting kernel firmware and cutting out inappropriate bloat?

[jamesbond]

xinput installed on puppy results in # xinput list commmand showing one mouse0 and keyboard0, the same command on L64 and all other linux distros results in a list of input devices as seen in hardinfo - input . a 'Master' is the head to which are attached the first keyboard and mouse and all other input devices. By creating a second master xinput allows you to attach to it a second keyboard and mouse which gives another pointer that acts indepedently of the first as in multi-touch. You can of course create several masters with several pointers and keyboards etc. It may be possible still to 'bind' a master to a display/screen/desktop. Which is useful for me and my family to share one pc with extra displays keyboard and mice = pseudo multi-seat without nested servers = accelerated graphics and much better performance. Not for most people's usage I know.

Ah, that's because xorg hotplugging is *NOT ENABLED* in puppy. The ones built with deb-build does work (if you also apply the pinstall.sh patches from rootfs). For others, what stemsee says is quite interesting, here's what Arch has to say about it: https://wiki.archlinux.org/index.php/Multi-pointer_X.
EDIT: typo.

@Mav, good that ZDRV is loaded into RAM
@Mick, (about -o64) I see, thanks. I use that "-o 64" because the default is "-o 0" (basically means that the start of the partition includes the partition table itself) and certain machines will not boot with this setup.

--if suffix is -nfg then you absolutely know it's no good

LOL

PS @jamesbond. re kernel-kit. Do you have a script for getting kernel firmware and cutting out inappropriate bloat?

No. What we usually do when we build the modules.sfs is something like this

Code: Select all

# in kernel source, after completion of make oldconfig
make -j6 bzImage modules
make modules_install INSTALL_MOD_PATH=./modules
cp -a /lib/firmware modules
mksquashfs modules kernel-modules.sfs -comp xz -Xbcj x86

Basically we copy whatever existing firmware in the running Fatdog, may be plus a few that others have reported missing in the forum. What we have in firmware is quite large and may contain obsolete stuff as well (I cleaned out radeon firmware before we released 630, I think, but there are many other that I haven't looked at) ...

[stemsee]

[quote="jamesbond"No. What we usually do when we build the modules.sfs is something like this

Code: Select all

# in kernel source, after completion of make oldconfig
make -j6 bzImage modules
make modules_install INSTALL_MOD_PATH=./modules
cp -a /lib/firmware modules
mksquashfs modules kernel-modules.sfs -comp xz -Xbcj x86

Basically we copy whatever existing firmware in the running Fatdog, may be plus a few that others have reported missing in the forum. What we have in firmware is quite large and may contain obsolete stuff as well (I cleaned out radeon firmware before we released 630, I think, but there are many other that I haven't looked at) ...[/quote]

Exactly what I did and thought should be the solution.

XINPUT on puppy: After maybe two years I have my answer! Thank you Jamesbond. Barry Kauler owes you $50 au ! Or half anyway. lol

Also I am now typing from the trusty built in woof-ce-ng. Gave a blowfish result of 8.1 Unheard of!!!