Kernel 2.6.27 and squashfs 3.4 patch problem [solved]

[MU]

test Iso:
http://www.murga-linux.com/puppy/viewto ... 771#239771

updated atheros madwifi driver for the eeepc:
http://www.murga-linux.com/puppy/viewto ... 167#242167

---------------------------
Hey peeps

I try to compile Kernel 2.6.27 in Puppy 4.1.

I made a SFS of the source from kernel.org.
Used my modified .config with smp support from Puppy 4.1.

Then applied these patches:

loglevel (see patches-required.txt).

patch -p1 </mnt/sdc7/kernel-2.6.27/squashfs3.4/kernel-patches/linux-2.6.27-rc4-next/squashfs3.4-patch
patch -p1 </mnt/sdc11/puptrix/sources/kernel-2.6.25.16/fsync_super-2.6.19.patch
patch -p1 </mnt/sdc11/puptrix/sources/kernel-2.6.25.16/deny_write_access.patch
patch -p1 </mnt/sdc11/puptrix/sources/kernel-2.6.25.16/unionfs-2.4_for_2.6.25.12.diff

Ran:
make oldconfig
(answered many questions without deeper knowledge)
make
During compilation I now get:

Code: Select all

   CC      fs/squashfs/inode.o
fs/squashfs/inode.c: In function 'squashfs_export_iget':
fs/squashfs/inode.c:647: error: implicit declaration of function 'd_obtain_alias'
fs/squashfs/inode.c:647: warning: assignment makes pointer from integer without a cast
make[2]: *** [fs/squashfs/inode.o] Error 1
make[1]: *** [fs/squashfs] Error 2
make: *** [fs] Error 2

The readme of the sqaushfs patch sais, that for 2.6.27 some things in the vfs might change, so linux-2.6.27-rc4-next shall be used, but it might not work.

resources:
squashfs version 3.4
http://sourceforge.net/project/showfile ... p_id=63835
homepage:
http://squashfs.sourceforge.net/

Did someone find a solution already?
Mark

[MU]

ok, I see that for 2.6.26.3 still Barrys modified patch shall be used, so I'll try that one now.

Use Barry's 4300_squashfs-3.3.patch kernel patch instead of the one included in the squashfs-3.3 package. And use Barry's instructions for patching Puppy 2.17 found in patches-required.txt.

http://www.murga-linux.com/puppy/viewto ... 824#234824

I do not use the lzma patch yet, to avoid even more potential errors.

Mark

[MU]

No, does not work either.
I started over again with Barrys patch.
Now I get:

Code: Select all

  CC      fs/unionfs/file.o
  CC      fs/unionfs/inode.o
fs/unionfs/inode.c: In function 'unionfs_symlink':
fs/unionfs/inode.c:416: error: too many arguments to function 'vfs_symlink'
fs/unionfs/inode.c: In function 'unionfs_permission':
fs/unionfs/inode.c:805: error: implicit declaration of function 'permission'
fs/unionfs/inode.c: At top level:
fs/unionfs/inode.c:957: warning: initialization from incompatible pointer type
fs/unionfs/inode.c:973: warning: initialization from incompatible pointer type
fs/unionfs/inode.c:984: warning: initialization from incompatible pointer type
make[2]: *** [fs/unionfs/inode.o] Error 1
make[1]: *** [fs/unionfs] Error 2
make: *** [fs] Error 2
# 

Mark

[big_bass]

Did someone find a solution already?

Hey Mark
looks like your a pioneer on puppy with what your doing

I did compile with squashfs support 3.4
using a different kernel the 2.6.24.5 with smp
and all works fine for me

note : that I didnt use Barry's config or any of his kernels
because for what I wanted to do they were not needed
so I present this as a squash3.4 patch only info


last post by me there
http://www.murga-linux.com/puppy/viewto ... &start=105

shows I got it working

so my point here is the patch does work
the problem is elsewhere


I will try to add the other patches also
and see how it goes I am curious
if they work on another kernel


big_bass

[MU]

I wonder if the unionfs patch is required, when we use aufs?

I currently compile without it.
And for squashfs I use not linux-2.6.27-rc4-next but linux-2.6.27-rc4.

Now it compiles all the files in the fs/ subfolder.

I am curious, if it still will be possible, to build the aufs module afterwards.

I will test that on monday.
I go asleep now while it continues to compile, and tomorrow (oops.. today) I am away in the forrest to collect mushrooms.
Mark

[MU]

ok, the kernel compiled now.
Aufs would not, but I found a patch for ubuntu, that at least allows to compile it.
https://lists.ubuntu.com/archives/kerne ... 03147.html

You can apply it like this:
Edit fs/aufs25/Makefile

around line 28

Code: Select all

# defined in ${srctree}/mm/shmem.c in linux-2.6.26 and earlier
# tristate
ifdef CONFIG_TMPFS
#MU ifeq ($(shell test ${SUBLEVEL} -le 26 && echo t),t)
ccflags-y += -DTMPFS_MAGIC=0x01021994
#MU endif
endif

So I commented a condition (#MU)

I applied that to the CVS version, but it might work in others, too.

I cannot quickly test the kernel, as I have a frugal installation.
It will take some time on monday to sort out, how to run depmod and patch initrd.gz to use the kernel, as now the foldernames change according to the new versionnumber.

I attach the .config I used.
It uses the usb hid and hostdrivers inbuilt, not as modules.
This was a request somewhere else to allow full installations on a usb-stick.
It also might reduce problems, when keyboards are not detected at startup.
It does not use lzma, as stated above.
The resulting bzImage (vmlinuz) is 2.3 MB, and there seem to be many more modules than in Kernel 2.6.25.16.

There is a lot of multimedia stuff, like gspca and dvbt drivers and so on.

**edit** updated the config, it now is the same as in the source SFS.
**edit** updated again, now with lzma patch
**edit, updated again to version 2 and 4, as version 3 was buggy.

Mark

[amigo]

I remember seeing something that Barry said about trying a recent kernel, but having to give up because the squashfs patch wouldn't work -that is it wouldn't apply cleanly.
When you apply any patch, use shift+pgup to scroll back throught the prompts to make sure the patch has applied okay -that is there are no failed hunks.

One of the real drawbacks to Puppy is its' reliance on so many third-party kernel patches. This wil always leabe you waiting for the author of the patch to update to the latest kernel version, or even leave you completely out in the cold. Sometimes chanegs are made in the kernel which require extensive re-writing of some patches.

Looking forward, Puppy should probably forget about sfs files altogether. I know you don't want to hear that, but it will always remain a sticking point. The emphasis on keeping to a certain size -very limited at that- will always force you to make choices which put you in these situations and others.

Now, about fiinding out which patches are used and which not, I also see very poor note-taking in Barrys' notes. One thing you can do is take his full patched sources and start reversing the indiviudal patches. If one completely fails to revers, that means it wa never applied.

I have a lot of experience working with lots of patches -it can be maddening when you don't even understand what the code does or is supposed to do. keeping up-to-date with developments in the 2.6 kernel series is only getting worse -not better. Each micro-version update since 2.6.24 has been a patch of over 100,000 lines. And each patch is a 'quilt' patch that applies over 1000 fixes, updates and new features, so it is nearly impossible to isolate all(and only) the code which pertains to a single feature or fix.

Redhat, and SuSE have an *army* of people who work on their kernels and these are some of the same guys who write code for the kernel. Unless Puppy can recruit some really talented folks to work on these things, a depp change of concept is needed -or the conviction to stay way off of the bleeding edge.

Oh, another tip. After running 'make oldconfig', run 'make menuconfig' to check and see if you have enabled all the options like you wanted.

[amigo]

I remember seeing something that Barry said about trying a recent kernel, but having to give up because the squashfs patch wouldn't work -that is it wouldn't apply cleanly.
When you apply any patch, use shift+pgup to scroll back throught the prompts to make sure the patch has applied okay -that is there are no failed hunks.

One of the real drawbacks to Puppy is its' reliance on so many third-party kernel patches. This wil always leabe you waiting for the author of the patch to update to the latest kernel version, or even leave you completely out in the cold. Sometimes chanegs are made in the kernel which require extensive re-writing of some patches.

Looking forward, Puppy should probably forget about sfs files altogether. I know you don't want to hear that, but it will always remain a sticking point. The emphasis on keeping to a certain size -very limited at that- will always force you to make choices which put you in these situations and others.

Now, about fiinding out which patches are used and which not, I also see very poor note-taking in Barrys' notes. One thing you can do is take his full patched sources and start reversing the indiviudal patches. If one completely fails to revers, that means it wa never applied.

I have a lot of experience working with lots of patches -it can be maddening when you don't even understand what the code does or is supposed to do. keeping up-to-date with developments in the 2.6 kernel series is only getting worse -not better. Each micro-version update since 2.6.24 has been a patch of over 100,000 lines. And each patch is a 'quilt' patch that applies over 1000 fixes, updates and new features, so it is nearly impossible to isolate all(and only) the code which pertains to a single feature or fix.

Redhat, and SuSE have an *army* of people who work on their kernels and these are some of the same guys who write code for the kernel. Unless Puppy can recruit some really talented folks to work on these things, a deep change of concept is needed -or the conviction to stay way off of the bleeding edge.

Oh, another tip. After running 'make oldconfig', run 'make menuconfig' to check and see if you have enabled all the options like you wanted.

[big_bass]

Hey Mark

To keep a log of this so If someone else runs into this problem
this will show what was done to apply the patch and build the tools on my slackware 12.1
install

***note :not built on puppy or using the puppy config so your directories will be different
but its just a reference to show that the latest 3.4 patch works *

root:~# cd /usr/src/linux-2.6.24.5
root:/usr/src/linux-2.6.24.5# patch -p1 < /usr/src/squashfs3.4/kernel-patches/linux-2.6.25/squashfs3.4-patch
patching file fs/Kconfig
Hunk #1 succeeded at 1405 (offset 38 lines).
patching file fs/Makefile
Hunk #1 succeeded at 72 (offset -1 lines).
patching file fs/squashfs/inode.c
patching file fs/squashfs/Makefile
patching file fs/squashfs/squashfs2_0.c
patching file fs/squashfs/squashfs.h
patching file include/linux/squashfs_fs.h
patching file include/linux/squashfs_fs_i.h
patching file include/linux/squashfs_fs_sb.h
patching file init/do_mounts_rd.c
root:/usr/src/linux-2.6.24.5#



-------------------------------------------
now the tools for squash so you can make a squashfs
or edit one


root:/usr/src/linux-2.6.24.5# cd /usr/src/squashfs3.4/squashfs-tools
root:/usr/src/squashfs3.4/squashfs-tools# make
cc -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -O2 -c -o mksquashfs.o mksquashfs.c
cc -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -O2 -c -o read_fs.o read_fs.c
cc -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -O2 -c -o sort.o sort.c
cc mksquashfs.o read_fs.o sort.o -lz -lpthread -lm -o mksquashfs
cc -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -O2 -c -o unsquashfs.o unsquashfs.c
cc unsquashfs.o -lz -lpthread -lm -o unsquashfs
root:/usr/src/squashfs3.4/squashfs-tools#


-------------------------------

that makes the mksquashfs and the unsquashfs



hope that helps better document what happens
with the patch and tools

big_bass

[amigo]

I've had a stab today at applying the patches to the latest 2.6.27 which is out. There are some pretty big changes in it in some key spots. All the patches expect the linux-lzma apply fairly cleanly, although I have 'rediffed' them so they apply without errors.

The linux-lzma is gonna take some work -I may be able to get it going, but that code is stuck right in the heart of the kernel. This patch is not the one that makes squashfs use lzma .it makes the kernel be compressed with lzma instead of gzip. (BTW bzImage does not mean the kernel is compressed with bzip2). Anyway, the very first code loaded by the bootloader is the code which decompresses the rest of the kernel to RAM. I can't promise that I can get this right -I've tried to find a working patch for 2.6.27 but no luck yet.

To give a little perspective to the problem -I downloaded the 2.6.27 patch -this thing is 80MB when decompressed and conatins over 1,500,000 lines!

I also had a look at the changes between the squashfs-3.4 patch versions for 2.6.26, 2.6.27-rc4 and 2.6.27-rc4-next. there is only one line different bewteen the first two. 2.6.27-rc4-next makes a few lines of change, but the associated changes in the surrounding kernle code apear to have been rolled back -this is why you could succesffully compile with the 2.6.27-rc4 patch applied against 2.6.27.

Anyway, the linux-lzma patch is obviously not essential to making Puppy work -it just means a smaller kernel image.

Mark, I'd suggest that, in general, you stick to working with the last stable kernel -in this case 2.6.26.6. Another even more conservative approach would be to use the alternate kernel bramch maintained by adrian Bunk -that's the 2.6.16 kernel with continous fixes. Adrian is not too happy with the fast pace of changes in the main 2.6 tree, so he decided a couple of years ago to start keeping a 2.6.26 tree updated in a more stable way -that is only fixes are apllied without all the re-coding and new features of the main trees. He is at 2.6.16.62 now. His tree is similar to what redhat and others do as far as providing upgrades to their products which they provie with long-term support. They don't go just upgrading everytime the kernel number gets bumped.

Still, I'll have look at getting the linux-lzma patch working for 2.6.27 -Hell, I don't even run 2.6 kernels -I run 2.4 kernels for my everyday use -i don't wanna spend my days in grief over the latest catastrophes in 2.6. I'm mean come on -1.5 million lines in a patch? Who expects thing to just work... Rant over1

[BarryK]

MU,
You should use the unionfs patch specifically designed for 2.6.27. You get it from the unionfs site. It's up to version 2.5 now.

[MU]

thanks.
As I had compiled without unionfs already, I continued my tests without it.
Was curious...

Looks good so far:

Code: Select all

# uname -r
2.6.27
# lsmod
Module                  Size  Used by
snd_mixer_oss          15104  0 
parport_pc             29892  0 
lp                     10500  0 
parport                34668  2 parport_pc,lp
snd_ens1371            21280  0 
gameport               11788  1 snd_ens1371
snd_rawmidi            21280  1 snd_ens1371
snd_seq_device          7948  1 snd_rawmidi
snd_ac97_codec         96928  1 snd_ens1371
ac97_bus                2688  1 snd_ac97_codec
snd_pcm                72196  2 snd_ens1371,snd_ac97_codec
snd_timer              20872  1 snd_pcm
snd                    52900  7 snd_mixer_oss,snd_ens1371,snd_rawmidi,snd_seq_device,snd_ac97_codec,snd_pcm,snd_timer
soundcore               7776  1 snd
snd_page_alloc          8968  1 snd_pcm
pcspkr                  3456  0 
natsemi                26848  0 
i2c_piix4               8720  0 
i2c_core               24212  1 i2c_piix4
intel_agp              25660  1 
agpgart                33352  1 intel_agp
evdev                  10144  0 
thermal                16028  0 
processor              33964  2 thermal
button                  6928  0 
fuse                   50972  0 
aufs                  283408  1 
nls_iso8859_1           4864  1 
nls_cp437               6528  1 
# cat /etc/puppyversion 
410# 
# 

There were some probems:
I compiled builtin, not as modules:
usb-hid
usb-ehci
usb-ohci
usb-storage

so the core usb libs.

When I booted, pup_410.sfs could not be found.
After running some commands in the initial ramdisk console, I found out two reasons.

1.)
init, line 227:

Code: Select all

 if [ $RETVAL -eq 0 ];then
  for ONE in $MODUSB
  do
   modprobe $ONE-hcd && USB="1"
   [ "$USB" = "1" ] && [ "$ONE" = "ehci" ] && USB="2"
  done
#MU
USB="2"
  if [ "$USB" != "" ];then 

I added USB="2" to fool Puppy to believe, that the USB-modules loaded (that now arte builtin!).

After a reboot, I could continue to boot after some seconds by typing
. /init
I had not tried that before, so am not sure, if this first patch is required.
So I added a delay now before the pup_410.sfs is searched:

2.) init, line 254

Code: Select all

#filesystems...
modprobe nls_cp437     #needed by windows filesystems.
modprobe nls_iso8859-1 #needed by linux filesystems.
modprobe $LAYERFS #unionfs or aufs.
modprobe fuse #for ntfs-3g driver.

echo "mu sleep 10" >/dev/console
sleep 10

if [ "$WAITUSB" = "yes" ];then #wait for device to register.
 #v3.94 Classmate laptop, needs more delay here... no, further down...

I had to add 10 seconds of delay:
sleep 10

Maybe smaller values work, did not check yet...
This seems to be a problematic part, as Barry added there himself several sleep conditions.

Now, how did I get so far?
I ran "make modules_install".
Then I extracted initrd.gz.
I renamed in it lib/modules/2.6.25.16 to 2.6.27.
Replaced the modules there, but did NOT gzip them.
Added the "aufs" module.
Copied all textfiles like modules.dep from /lib/modules/2.6.27 (nothing from the subfolders, just these few textfiles).
Edited modules dep, added this in the beginning:

Code: Select all

/lib/modules/2.6.27/kernel/fs/aufs/aufs.ko:

Applied my patches to init as described above.

Now I could boot with the new vmlinuz and initrd.gz.

But when I typed "lsmod", almost no module was loaded.
/lib/modules/2.6.27 now was the same as in initrd,gz, so all my installed modules were gone.
There is no folder "initrd" there!
So there is an error in layering it to the final filesystem.

I then have rebuilt the modules, and repackaged pup_410.sfs with all of them included.
After reboot, now all are there, though there still is not this initrd folder.

Maybe this is, because I compiled without unionfs?
So though it works now with all modules, something still is wrong.

I will try to sort that out.
Mark

[MU]

ok, found out where the "initrd" folder went.

init has this code around line 194:

Code: Select all

#does this initrd have all the zdrv components inside it?...
ZDRVINIT='no'
[ `wc -l /lib/modules/$KERNVER/modules.dep  | tr -s ' ' | cut -f 2 -d ' '` -gt 200 ] && ZDRVINIT='yes'

Now as I copied the whole modules.dep to initrd.gz, this condition gives a wrong result.
So in that modules.dep all lines must be deleted, that have no modules.
It might be easier to use the one From Puppy 410 original, and just replace the kernelversion in the folderpath.

To test this, I edited instead init like this:

Code: Select all

#does this initrd have all the zdrv components inside it?...
ZDRVINIT='no'
[ `wc -l /lib/modules/$KERNVER/modules.dep  | tr -s ' ' | cut -f 2 -d ' '` -gt 200 ] && ZDRVINIT='yes'
#MU
ZDRVINIT='no'

As Puppy 4.1 no longer uses zdrv, this should be ok.

But I wonder:
WHY is required to have:
/lib/modules/2.6.27/initrd
Could the modules not be copied to the "proper" location?
So aufs would go to:
/lib/modules/2.6.27/kernel/fs/aufs/aufs.ko

For a frugal installation, that looks more logic.
Is it a relic from old Puppys?
Or is it required for special situations?
-----------------------------------------

With this change, we however now have back the folder /lib/modules/2.6.27/initrd.
However, the "real" modules.dep does not use it yet, as it uses the modules in the "proper" location.
So now we could delete again the "proper" modules from the pup_410.sfs, that had been copied to initrd.gz.
This would reduce pup_410.sfs by some kb, and we would use Puppys original concept.
Then we must run:
depmod -a
Now we would have a "final" modules.dep, that we could use in pup_410.sfs.

This is "almost" perfect, now we also should gzip again the modules in initrd.gz, and finally edit both modules.dep files correspondingly (for these few modules use .ko.gz instead of .ko).

I wonder, if it would make more sense, to build ALL modules from initrd.gz "builtin".
This would simplify Kernel updates, as initrd.gz no longer had to be edited/modified.
The ones in initrd.gz currently are 716 kb (NOT gzipped, or 272 kb gzipped).
So they would not make vmlinuz much larger.

------------------------------
Two things remain:

1.) the delay I had to add mentioned above - could this be replaced with some code instead, that checks, if everything is loaded?
Some forum messages indicate, that also the unmodified Puppy might have issues here (pup_410.sfs not found).

2.) my new Puppy does not include any 3rd party drivers (like network).
Those had to be compiled, too.

----------------------------------
I will make some more tests tomorrow, and then upload a Puppy 4.1 with Kernel 2.6.27.
Then others can look too at these issues using that testsystem.

Mark

[MU]

I uploaded a test Iso.
ftp://ks301128.kimsufi.com/testing/puppy4.1-k2.6.27/

I reduced the delay mentioned above to 4 seconds.
I modified /etc/rc.d/rc.sysinit.
Here I removed the check for loaded USB modules and yenta_socket.
ALL modules except aufs.ko.gz, that formerly were in initrd.gz, now are inbuilt into the Kernel.
Unionfs patch is not included, but aufs is default, so this should not matter. Especially as unionfs is very buggy.
SMP is activated.

NO 3rd party drivers are included!

I also uploaded the Kernel source, also pre-compiled.
Have a look at the readme.txt.

I do NOT plan to compile the 3rd party drivers from Puppy (too busy with Muppy now), but you could do that with the sources from:
ftp://ks301128.kimsufi.com/Puppylinux-s ... 2.6.25.16/

Not all might be needed, as this new kernel should support more devices by default (e.g. the gspca webcam drivers are already included).

If you compile additional drivers, please attach them here.

Please let us know, if/how this versions works for you.

Mark

[jamesbond]

Mark,

Does this iso contains all the updated modules? Or are they mixed modules - some from 2.6.25.16 and some from 2.6.27 ?

cheers!

[MU]

It includes only the modules that came with the Kernelsource of 2.6.27.
I removed all modules and firmware from 2.6.25.16.

wait... here is a list:
ftp://ks301128.kimsufi.com/testing/pupp ... odules.txt
Mark

[jamesbond]

Thanks.

It works fine on my desktop - ethernet works, sound works, video works, external usb harddisk works, and both my CPUs are detected. No glitch at all. I only tested live CD though. Thanks for this.

Hardinfo report attached.

EDIT: just to note that this post is made with your 2.6.27 iso.
EDIT: this kernel even enables me to do something i haven't been able to do in a very long time - watch a VCD !

[big_bass]

I uploaded a test Iso.
ftp://ks301128.kimsufi.com/testing/puppy4.1-k2.6.27/



Mark

Hey Mark great to hear you got a test iso
so quickly

I am downloading
and will give it spin too


big_bass

[aragon]

hi mu,

to give you a short response on my testings:

Laptop Lenovo 3000 N200 Intel(R) Pentium(R) Dual CPU T2370 @ 1.73GH

- video => works out of the box (normal 4.10 too)
- sound => works out of the box after alsa_config (first time ever on linux no 'hand work'=> snd_hda_intel)
- wlan => does not work at all (normal 4.10 does) see attached pic. the failure occurs by running the connect-wizard. the loaded iwl3945 seems to be the correct one (same as in 4.10 'normal').

if it helps i have uploaded a hardinfo-report of both to

http://www.mediafire.com/aragon-puppy in test-2627

maybe we could fix the problem, as i'm too very interested on using dual-core.

cheers
aragon

[MU]

aragon
It seems I forgot to update /lib/firmware.
Please install this attached pet.
ftp://ks301128.kimsufi.com/testing/pupp ... rmware.pet

I updated the iso, too, it now includes these updated firmware files.

Mark