How to recompile Puppy 4.1 kernel with SMP support?

Using applications, configuring, problems
Message
Author
User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#21 Post by MU »

yes, 4.1 alpha6 works.
Checking 'hlt' instruction... OK.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 14k freed
ACPI: Core revision 20070126
ACPI: setting ELCR to 0200 (from 0418)
CPU0: Intel Pentium III (Coppermine) stepping 03
SMP motherboard not detected.
Local APIC not detected. Using dummy APIC emulation.
Brought up 1 CPUs
So why not my selfcompiled 4.1 with smp and aufs :cry:
Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

kirk
Posts: 1553
Joined: Fri 11 Nov 2005, 19:04
Location: florida

#22 Post by kirk »

Mark,

I'm running 2.6.26.3 with aufs and smp on a freugal install. (I only have a single core processor). I didn't use the unionfs or lzma patches. Have you tried booting with loglevel=7? You did depmod your new initrd.gz? You probably tried that stuff, just grasping at straws.

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#23 Post by MU »

kirk,
no, depmod was not needed, as I used the same kernelversion.

And loglevel=7 ... just did not remember that.

But I now found out my bug.
I had to patch aufs :!:
Barry explained it in modules-compile.txt, but that was the confusing part.

All I had to do:

extract 3rd-party-drivers/filesystem/aufs-20080825.tar.gz
Then edit the file in it: local.mk
Only one line changes:

Code: Select all

export CONFIG_AUFS_UNIONFS23_PATCH =
to:

Code: Select all

export CONFIG_AUFS_UNIONFS23_PATCH = y
Then compile the module again with:
make -f local.mk

I then added it again to initrd.gz, and BINGO! works :P :P :P :P :P

I uploaded a corrected ISO.
IMPORTANT: read:
ftp://ks301128.kimsufi.com/testing/pupp ... readme.txt

I also added a readme.txt and the SMP.config
I can write a step by step instruction tomorrow to summarize this thread, I'm too tired now.

Kirk, many thanks for your great summary, without it, I never would have had the courage to start the adventure of Kernel-compiling...
Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

Bruce B

#24 Post by Bruce B »

Mark,

Thank you for the history on logistics.

There wasn't any mention of fundamental regard and respect for others in your history. This is not to imply, I think these things aren't of importance to you. I do however think they are a critical part of successful logistics. So I'd like to add thoughts.

First of all, as you know, kernel software is different other software. It can be very distribution and version specific and likely is. To this extent we rely on the developer's fully pulling through for us and giving us the software we need.

Developer responsibilities are all very clearly stated. Albeit, there are people who understand FOSS and the licenses better than others.

I know the proper place to download the kernel source code. That is where I downloaded binary. The kernel I download from ibiblio.org compiled beautifully and failed miserably. The reason why is, even though the version matched the Dingo version, it was not the right source code.

I do not buy for one minute there are any logistical problems in merely adding a text file, explaining what that kernel could and couldn't be used for. Thus my consideration about lack of respect and consideration, by way of omission. Nobody could divine this failure, it was a stumbling stone waiting to happen.

At a minimum this should have been explained. Moreover, after my experience, there have been others doing the correct and logical thing, thinking that was the appropriate source code. On the forum, my responses were clearly that it will never ever work.

It's not my job to do this, although I'm more than pleased to help others avoid the pitfall I feel into.

Another thing is, if the proper download location doesn't contain the complete kernel source code. (for all versions released over the last three years). A text file or html file showing the proper locations is a minimum showing of good faith on the part of the developer.

I don't imagine it meets the license requirements, it does however
show good faith.

What we actually have, at present, is a puzzle to solve locating the kernel source code. The reason for the puzzle is simple omission of information at the proper and designated place.

When you, Barry and others work out logistics, I suggest you factor in fundamental respect, in the area of making the proper and complete communications in the right places.

I would also consider that Puppy has an irrevocable license to the source code. Not one human being in this planet can take this license from Barry. What more could a person ask for? Millions of dollars of software for free which is utterly irrevocable.

Only one person can terminate that license and it is Barry. He would do it automatically, simply by not complying with the conditions of the license. Willfully or otherwise. Although, the perceived willful nature will have a lot to do with how much wiggle room he gets. I would never play loose and free with that collection of code.

Do not mistake terminated with suspended. Terminate your own license by your own actions and you can find yourself at mercy of any kernel copyright holder instructing you to remove his code from the kernel. And you will remove it.

And if anyone wants to make the simple complex, then I'll file a violation complaint to see to it that the complex gets put back simple. This means there were be zero mystery where the source code is located. No more puzzles for anyone. The GNU is designed to be simple, straightforward, and keep things simple. Let's keep it that way.

I hope as you work out your logistics with Barry and others, and you keep in mind my post. What I really want is for you to be fully educated and read the license for yourself. It's about eight grade English material and a short read. Very simple for you, even though English is not your first language. Also of interest is the violation report questions. I will show you exactly what the attorneys are looking for in terms of enforcing violations.

You are a good guy, don't find yourself making problems, when you only motive is to help. Be educated in this area, this way your logistic planning will work out as intended.

Best wishes,


Bruce

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#25 Post by BarryK »

As I had pre-announced on my blog, the patched kernel sources for 4.1 and 4.1retro, in the form of SFS files, will be uploaded, and that is now done, at ibiblio, same place as the binary live-cd.
[url]https://bkhome.org/news/[/url]

User avatar
erikson
Posts: 735
Joined: Wed 27 Feb 2008, 09:22
Location: Ghent, Belgium
Contact:

#26 Post by erikson »

MU wrote:I now compiled the kernel in Puppy 4.1.
The compilation itself takes 4 hours, then more time is needed, to replace the modules by hand in initrd.gz.
Out of curiosity: on which hardware?

Another question, to all: would it be possible to add usb support to the kernel, to allow booting Puppy full installs off of usb devices such as external hdd (cfr http://www.murga-linux.com/puppy/viewtopic.php?t=34263)? What does it take to do so?

My reason for asking: gparted shows some "unallocated space" on the internal hdd of my laptop (which is not unexpected since it's an hp/compaq), so I'm very reluctant to change its partitioning (now ntfs single partition with WinXP) and I'm unable to do a full install. It's a pity, considering that my external usb hdd has plenty of space for compiling, experimenting etc without risk for my WinXP setup.
[size=84][i]If it ain't broke, don't fix it.[/i] --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
[url]http://www.desonville.net/[/url]
Puppy page: [url]http://www.desonville.net/en/joere.puppy.htm[/url][/size]

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#27 Post by MU »

I compile on a Pentium III 700 mhz with 1 gig of Ram.
On a modern athlon with smp it should be much faster, less than an hour I think.
For me, smp is important, as I often build huge squashfs files.
It can take up 1-2 hours to create a new Muppy.
So now that smp works, I can soon go and buy a modern computer.
Then I can take full advantage of the high speed, as mksqashfs supports multiple processors.

Concerning usb I am not certain.
Look at /usr/src/linux-2.6.25.16/.config
Here is the one with smp:

ftp://ks301128.kimsufi.com/testing/pupp ... SMP.config

You will see such lines:

Code: Select all

#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
You could try to use instead:

Code: Select all

CONFIG_USB_EHCI_HCD=y
So it would not be a module, but inbuilt.
There are not many, look in
/lib/modules/2.6.25.16/initrd/kernel/
The usb modules there you might try to compile with "y" instead of "m".

I have no idea, if that would work, but it may be worth a try.

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
erikson
Posts: 735
Joined: Wed 27 Feb 2008, 09:22
Location: Ghent, Belgium
Contact:

#28 Post by erikson »

MU wrote:I compile on a Pentium III 700 mhz with 1 gig of Ram.
Okay, then my laptop should do as well (1.6GHz / 480MB+swap)
I have no idea, if that would work, but it may be worth a try.
Surely. I know which modules are needed (I poked around in init), so it would be just a matter of enabling them in the config file.

Now the problem is to get it compiled... I understand from kirk's comments (thread "compiling 2.6.26 kernel" IIRC) that compiling the kernel "can" be made to work on frugal install (though Barry recommends full install), but "not" on a usb drive. Chicken and egg problem :-(
[size=84][i]If it ain't broke, don't fix it.[/i] --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
[url]http://www.desonville.net/[/url]
Puppy page: [url]http://www.desonville.net/en/joere.puppy.htm[/url][/size]

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#29 Post by MU »

I think it might work on a stick.
It is a full installation on the stick?
So you just had to extract the source there as I described it in the beginning, and creatze the symlink.
My first message should provide enough info for the start.
Once you got as far as me, I can help you further.
I slept bad and am unconcentrated, so I will write a full instruction with the additional steps tomorrow.

But it basically will be the same as Kirks instruction.
Mine just will be very specific for Puppys Kernel (so a bit simplified), while Kirks provides info for general Kernels.

Markk
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
erikson
Posts: 735
Joined: Wed 27 Feb 2008, 09:22
Location: Ghent, Belgium
Contact:

#30 Post by erikson »

MU wrote:I think it might work on a stick.
It is a full installation on the stick?
Same problem with usb stick, Mark. Can't run full-install from usb stick as long as the kernel has no usb driver support compiled in :-)

For the same reason one probably can't compile the kernel from a Puppy on stick either. Actually I'm not quite sure why kernel compilation off usb stick/hdd would not work, if such compilation can be made to work from frugal installs (as per Kirk's remarks).

Anyway, I first have to download the source, read the relevant threads again, and do some experimenting.
[size=84][i]If it ain't broke, don't fix it.[/i] --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
[url]http://www.desonville.net/[/url]
Puppy page: [url]http://www.desonville.net/en/joere.puppy.htm[/url][/size]

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#31 Post by MU »

so it is a frugal installation on the stick?
So I wonder, what the problem is.
Is it possible to use .sfs files?

If no, one still could mount devx_410.sfs "by hand", and copy the contents to the savefile.

Or use SFS-Combiner to merge devx_410.sfs with pup_410.sfs.
Same with the 2.6.25.16_410.sfs or Barrys new kernel sfs.

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

Bruce B

#32 Post by Bruce B »

Mark,

Four hours? I think processor speed and ram matter. It would be nice if a kernel compile gave me time to brew up a pot of coffee, but as it is, I might be able to microwave a cup of instant coffee.

Truthfully, I love watching the output of the compile and wouldn't leave for coffee. But four hours, is a long time to watch output. Probably nothing we'd ever be able to sell as a movie and get rich on.

Please keep us posted. I have held back offering testing anything, until I know you have all the right software to work with.

When you think things are right and ask for testers. Please be specific about any and all things you want tested.

Bruce

User avatar
erikson
Posts: 735
Joined: Wed 27 Feb 2008, 09:22
Location: Ghent, Belgium
Contact:

#33 Post by erikson »

MU wrote:So I wonder, what the problem is. Is it possible to use .sfs files?
I can use .sfs without problems. I also can compile.

The point is that, for compiling the kernel,
(1) Barry recommends using full install (cfr http://www.puppylinux.com/blog/?viewDetailed=00404),
(2) Kirk says that it's nogo from usb drives (cfr http://www.murga-linux.com/puppy/viewto ... 637#234637).

If I understand correctly, the issue is not just compiling the kernel, but the other stuff required to make a recompiled-kernel Puppy work (such as generating the module dependencies list).
Bruce B wrote:Truthfully, I love watching the output of the compile and wouldn't leave for coffee. But four hours, is a long time to watch output. Probably nothing we'd ever be able to sell as a movie and get rich on.
I vividly recall the times when it took me a full working day, on three Intel MDS development systems working in parallel, for generating an 8085 application program of roughly 32 KB (kilobytes) object code (some 25,000 lines of macro-assembler source code). Today's SW developers are really spoilt :-)
[size=84][i]If it ain't broke, don't fix it.[/i] --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
[url]http://www.desonville.net/[/url]
Puppy page: [url]http://www.desonville.net/en/joere.puppy.htm[/url][/size]

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#34 Post by MU »

If I understand correctly, the issue is not just compiling the kernel, but the other stuff required to make a recompiled-kernel Puppy work (such as generating the module dependencies list).
yes, this also applies to frugal.
But you can skip the "depmod", if you use the SAME kernelversion.
Like this you also can use frugal, and it should work for your USB-Stick, too.


Compiling the Puppy Kernel with new options using a frugal installation

You boot Puppy with devx_410.sfs and the sfs with the Kernelsource.
For the following instructions, I used the Kernelsource:
ftp://ks301128.kimsufi.com/testing/pupp ... 16_410.sfs
Barry meanwhile released a official one, so you might prefer that one:
http://www.puppylinux.com/blog/?viewDetailed=00407
There you also find devx_410.sfs.

You then must save
ftp://ks301128.kimsufi.com/Puppylinux-s ... 8-STANDARD
as
/usr/src/linux-2.6.25.16/.config
You now could edit it to your needs.
If you just want to add SMP support, you can use the config, that already includes the required changes:
ftp://ks301128.kimsufi.com/testing/pupp ... SMP.config

Now you compile the Kernel:


cd /usr/src/linux-2.6.25.16/
make oldconfig
make
make modules_install


The last step installed the modules to the running system.
Then you copy /usr/src/linux-2.6.25.16/arch/x86/boot/bzImage
as "vmlinuz" to the place, where your old vmlinuz is, that boots Puppy.

You cannot reboot yet, though, because initrd.gz still has the old modules.
So you must replace them.
If you now had a Kernel with a different versionnumber, you also had to do additional steps.
You had to run depmod on initrd.gz, what is somewhat tricky.
Kirk explains that more detailed:
http://www.murga-linux.com/puppy/viewto ... 637#234637
So I used the original kernelversion, as like this, I can skip this step.

Before you continue, please read further down: The aufs driver
Maybe that could be done here already, so we had to rebuild initrd.gz only once!


So you just must update the modules in initrd.gz, I did this:
copied it to /root/test/
Saved there this script as "editinit":

Code: Select all

#!/bin/bash
gunzip initrd.gz
mkdir puppy-init 
cd puppy-init
cat ../initrd | cpio -d -i
leafpad init
find . | cpio -o -H newc | gzip -9 > ../initrd.gz
cd ..
rm -rf puppy-init
rm -f initrd
Made it executable and ran it:
cd /root/test
chmod 755 editinit
./editinit


Now leafpad opens, with "init".
If you close leafpad, the script continues to package initrd.gz again.
So you must NOT close leafpad yet!
You now will find the extracted initrd.gz as
/root/test/puppy-init/

It has an own folder /lib/modules/.
Examine it, and you will find for example
lib/modules/2.6.25.16/kernel/crypto/cbc.ko.gz

So you must copy there the new /lib/modules/2.6.25.16/kernel/crypto/cbc.ko

then type:
cd /root/test/puppy-init/lib/modules/2.6.25.16/kernel/crypto
rm cbc.ko.gz
gzip cbc.ko


Do the same with all other modules, that you can find there.

Finally close leafpad, so that the new initrd.gz is created.

Copy it to where your vmlinuz is stored, that boots Puppy.

Now reboot with the option "layerfs=unionfs".
You can add this in Grubs menu.lst as Kerneloption.
We need this, as we did not yet compile the new aufs module.

--------------------------------
The aufs driver
Finally we need to add the updated aufs driver.
I am not certain, if this must be done afterwards, as I describe here, or if it could be done already after "make modules_install".
I will check that, when I compile the next kernel.

extract somewhere ftp://ks301128.kimsufi.com/Puppylinux-s ... 825.tar.gz.
Then edit the file in it: local.mk
Only one line changes:

Code: Select all

export CONFIG_AUFS_UNIONFS23_PATCH =
to:

Code: Select all

export CONFIG_AUFS_UNIONFS23_PATCH = y
Then compile in that folder the module with:
make -f local.mk

Now also add fs/aufs25/aufs.ko to initrd.gz in /root/test/ as described above.
Then copy this new initrd.gz to the final location again, and reboot as usual (without the option "layerfs=unionfs").

Done.
You now could compile the additional 3rd party drivers, if the existing ones no longer work.
You'll find them here:
ftp://ks301128.kimsufi.com/Puppylinux-s ... 2.6.25.16/

-----------------------------
Sidenotes:

1.) If you use Muppy0084c3, then you must install this PET before compiling the kernel:
http://distro.ibiblio.org/pub/linux/dis ... ma-458.pet
Without it, vmlinuz is packaged wrong, and only 16 kb in size.

2.) You can find a recompiled Puppy with SMP for tests here:
ftp://ks301128.kimsufi.com/testing/puppy4.1-smp/
Read the readme.txt there for notes!

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
Brandon
Posts: 111
Joined: Sat 15 Dec 2007, 01:10
Location: New York
Contact:

#35 Post by Brandon »

MU, I downloaded your ISO, and it worked great! If all the modules were compiled, we would have an SMP Puppy!

Anyways, I added a hardinfo_report so we all can see the second core detected. (hardinfo_report2 is from regular Puppy 4.1)

PS: It seemed very fast.
Attachments
hardinfo.zip
my reports
(7.91 KiB) Downloaded 434 times
"Develop for it? I'll piss on it." -Bill Gates on the NeXTcube

User avatar
erikson
Posts: 735
Joined: Wed 27 Feb 2008, 09:22
Location: Ghent, Belgium
Contact:

#36 Post by erikson »

MU wrote:Compiling the Puppy Kernel with new options using a frugal installation
Great, thx for this detailed how-to.
You boot Puppy with devx_410.sfs and the sfs with the Kernelsource.
Meanwhile I have downloaded kernelsource and devx_410 from ibiblio, and peeked some into the sfs files.

Re. devx_410.sfs I notice that it contains hundreds of orphaned symlinks, in subdirs
./lib
./opt/qt4/lib
./usr/lib
./usr/lib/sane
./usr/X11R7/lib

Doesn't that cause difficulties?

I also notice that, for a frugal install, devx_xxx is "under" pup_xxx in the union, while for a full install, it's copied "over" the pup_xxx filestructure and thus takes precedence (while also the orphaned symlinks are resolved, I guess). Maybe that's why Barry recommends full-install for compiling the kernel?

Finally, I notice that ibiblio has only one devx_410 while there are two 410 versions (with 2.6.25.16 resp. 2.6.21.7 kernel). Is the same devx_410 applicable to both?
[size=84][i]If it ain't broke, don't fix it.[/i] --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
[url]http://www.desonville.net/[/url]
Puppy page: [url]http://www.desonville.net/en/joere.puppy.htm[/url][/size]

User avatar
MU
Posts: 13649
Joined: Wed 24 Aug 2005, 16:52
Location: Karlsruhe, Germany
Contact:

#37 Post by MU »

I notice that ibiblio has only one devx_410 while there are two 410 versions (with 2.6.25.16 resp. 2.6.21.7 kernel). Is the same devx_410 applicable to both?
yes, as the kernel is in a second sfs.
devx_410.sfs contains only common, kernel-independant files.
Re. devx_410.sfs I notice that it contains hundreds of orphaned symlinks, in subdirs
Ithink those point to the corresponding files in Puppy itself (that are in pup_410.sfs).
I encountered no problem in compiling the kernel because of that.
I used Puppy 4.1 only for my kernel tests, usually I run Muppy, that has an own devx.

Mark
[url=http://murga-linux.com/puppy/viewtopic.php?p=173456#173456]my recommended links[/url]

User avatar
erikson
Posts: 735
Joined: Wed 27 Feb 2008, 09:22
Location: Ghent, Belgium
Contact:

#38 Post by erikson »

Mark,

Please add explicitly to your last instructions that the lzma-458 pet must be loaded before compiling. Otherwise one ends up with a 16KB bzImage, as you found out earlier, and now me too.

BTW compile time is roughly 90 minutes on my 1.6GHz laptop.
[size=84][i]If it ain't broke, don't fix it.[/i] --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
[url]http://www.desonville.net/[/url]
Puppy page: [url]http://www.desonville.net/en/joere.puppy.htm[/url][/size]

purple_ghost
Posts: 416
Joined: Thu 10 Nov 2005, 02:18

Down side of booting SMP support in a single core computer?

#39 Post by purple_ghost »

Down side of booting SMP support in a single core computer? Is loss of RAM? Sometimes application programs run off the tracks?

Can there become problems in switching between the two cores? Which an actually cause some type of performance delay?

Anyone guess how much one gains in speed by using SMP support in a dual core machine?

I would feel that a derivative of Puppy, can have a long future as a web surfing machine. A version of Puppy that already has Java, and .... already installed for a newbie to use. Surely it is easier to use -- blank --version of Puppy with a live CD than it is to do all the things Windows now requires to keep malware scraped out of it. Gee Puppy is smaller and feels faster than Norton Antivirus alone.
Google Search of Forum: http://wellminded.com/puppy/pupsearch.html

User avatar
erikson
Posts: 735
Joined: Wed 27 Feb 2008, 09:22
Location: Ghent, Belgium
Contact:

#40 Post by erikson »

Results of today:

Using Puppy 4.1-k2.6.25.16 I recompiled the kernel from Barry's source, with Barry's original config. For various reasons it took me a couple of retries "from scratch".

I only did the 'make oldconfig' and 'make' which produced bzImage (with the lzma pet installed it's now 1589 KB i.e. nearly same size as original vmlinuz). I did *not* 'make modules_install' and did *not* update modules in initrd.gz - my reasoning is that these "ought" to be the same since I "self-compiled". I also did not yet recompile aufs.

I copied bzImage, renamed it to vmlinuz and rebooted (with unionfs).

Lo and behold, Puppy boots with the recompiled vmlinuz.

So the next step is to adapt the configuration, to include usb driver support, and see if my goal is achieved (boot full-install off external usb hdd).
[size=84][i]If it ain't broke, don't fix it.[/i] --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
[url]http://www.desonville.net/[/url]
Puppy page: [url]http://www.desonville.net/en/joere.puppy.htm[/url][/size]

Post Reply