Light-Debian-Core-Live-CD-Wheezy + Porteus-Wheezy

[saintless]

Hi, all.
Nothing more from me to add in Utilities and HowTo threads. Anything new you can find or suggest or questions if something is not clear enough just post it there as next post information.
I will focus most on testing DebianDog from now.

Toni

[fredx181]

Hi Toni,
Thank you!!!
Very clear explanation.
It's nice to have it this way with the choice for dpkg registration or not and the info about it, we should cooperate more often
The only thing I changed now is the size of the window so it shows all info.

rm -rf "/mnt/$DRV/$WRKDIR"$HOME/.fltk
rm -f "/mnt/$DRV/$WRKDIR"$HOME/.local/share/recently-used.xbel
rm -rf "/mnt/$DRV/$WRKDIR"$HOME/.local/share/Trash

Good additions, I first thought it wouldn't work for puppy user but it does.
It seems that the ktsuss program does that because when using "su - root" the outcome of 'echo $HOME' is /root.

There is one more option here which is the best one - some script for the future that will auto separate the packages from /var/lib/dpkg/info in new status and available and auto-make update-script.

Could you explain a little more about what that magic script exactly should do? It's a bit confusing for me.
I've been searching on the net for a script that reads from /var/lib/dpkg/info and transform in a status file but thats more for when dpkg is broken because of status file missing.

Edit: Just found this one here:
http://tuxx-home.at/archives/2005/02/16/T13_36_14/
It seems to work well and the path variable can be changed to live/cow/..... so it reads from the /live/cow/var/lib/dpkg/info (just guessing at the moment, not sure if this could be useful).
It needs gawk installed,btw.

Fred

[saintless]

There is one more option here which is the best one - some script for the future that will auto separate the packages from /var/lib/dpkg/info in new status and available and auto-make update-script.

Could you explain a little more about what that magic script exactly should do? It's a bit confusing for me.
I've been searching on the net for a script that reads from /var/lib/dpkg/info and transform in a status file but thats more for when dpkg is broken because of status file missing.

Hi, Fred.

Just finished this today. You will find the basic idea in this post:
http://ns1.murga-projects.com/puppy/vie ... 1e5#774461
But automating this is very complicated. I see automate script to do the following actions inside the separate module before making new squashfs:

1. Read the package name inside /var/lib/dpkg/info from package-name.list (skipping .list and package-name as name to search).
2. Find the package name and mark the package section information (maybe impossible to be done because the section is different number of lines for every package) inside status and available files.
3. Create status-(name of the new created module) and available-(name of the new created module).
4. Copy the marked sections form status and available inside status-(name of the new created module) and available-(name of the new created module) and leave one empty line between the packages and 2 empty lines at the end of the file.
5. Delete status and available from the working dir.
6. Create /opt/bin/(name-of-module)-executable-script
7. Add inside this script:

Code: Select all

#!/bin/bash

cd /var/lib/dpkg
cp -f status status-old
cat status-(name of the new created module) >> status
rm status-(name of the new created module)
cp -f available available-old
cat available-(name of the new created module) >> available
rm available-(name of the new created module)
cd /opt/bin
rm -(name of module)-executable-script 

Not sure at all if it can be done to work well. This is only idea to think about for the future.

Toni

Edit: Yes, this looks like something what we need:
http://tuxx-home.at/archives/2005/02/16/T13_36_14/
I have to test this proper but I think only updating status will be enough. available contains information for all installed on the system packages ever. many of them are removed but the information in available stays. The important one is status.

[fredx181]

Hi Toni,

Just finished this today. You will find the basic idea in this post:
http://ns1.murga-projects.com/puppy/vie ... 1e5#774461
But automating this is very complicated.
.......
.......

I knew it would be complicated, but it's soo... complicated!
I need a much more clear head (than I have now) to understand.
Will read more later.
Just a simple question:
What's the goal?
I think it's something like: Every module could have it's own "place" in the system and there's also no space wasted with duplicate libraries or binaries. Hope some of this is right, so then at least I understand something
Tell me if you think I can help with some bash scripting or something.

Fred

[saintless]

Hi, Fred.

It is not clear enough for me as well yet
What we gain with this is to have the same option for every module as every separate kernel module has - option to update dpkg if the user decide to remaster the system instead using always this module separate. In the case with changing kernel it is very important option but not so important for every package.
I also see some risk to give this option with RemasterCow because if some of the packages included inside this separate module are already installed as dependency for another package, then updating status file will put 2 entries for one package and this will break dpkg. This means to make the script much more complicated by adding one more step - searching in status file if some package is already installed (in case the module is created on fresh DebianDog and the user wants to add it on later remastered DebianDog). I will stop here
As I said it is not clear enough for me also yet. I just include some stuff I don't want to forget in HowTo thread for later use.

Do not think about this. Having disable dpkg registration default option is fine and safe method for creating module from changes. I'm happy with that.

BTW I need to fix blkid command to be included in the path when using sudo and su for puppy account in jwm-icewm version. I will add it in the first page fixes.

I will upload new RemasterCow tomorrow morning with pictures and post link to it in the beta thread.

Toni

[saintless]

Hi, Fred. Just for info.

Edit: Just found this one here:
http://tuxx-home.at/archives/2005/02/16/T13_36_14/
It seems to work well and the path variable can be changed to live/cow/..... so it reads from the /live/cow/var/lib/dpkg/info (just guessing at the moment, not sure if this could be useful).
It needs gawk installed,btw.

Thanks to this guy the most difficult part is solved. I will test the script proper to see how it works. It will be nice to include it as restore broken dpkg database option.
I will try to make separate script executing the above actions I wrote in extracted module to see how it goes. But I think separate tool for this (extract ready module and pack it back) will be better instead including all this in RemasterCow.

Toni

[mcewanw]

Yes :) exactly the same for me, it's not so damn slow to compress and btw, with it the system runs smoother, boots up faster, I'm beginning to wonder why we use xz compression!
For that few MB's less size? (yes, I know, more then a few, but AFAIK that's the only real advantage of xz )

Hi, Fred.
The only advantage of XZ is the smaller size and we pay for it with a little bit slower boot, but I doubt many users will experience slower boot or any difference from gzip compression. For Example William did not notice any difference except growing up the module size. I guess it depends on the hardware.
For JWM version the size difference is 32Mb. It counts for Copy to RAM option.

I didn't notice a difference in my one test of gz versus xz when it came to mplayer performance (basically monitoring its cpu load). However, I am concerned in case xz compression might be causing stability or timing problems on some machines, such as mine. It may be fine on fast enough machines, but I've been experiencing some very odd effects with CPU load occasionally being 100% on DebianDog and I have really struggled trying to find out why I'm generally experiencing too high a CPU load with installed mplayer on jwm DebianDog. It is using near 50% cpu on both my machines, which is far too much and restricting what other apps can be running at the same time. Same poor result with Fred's gnome-mplayer when I try it with jwm DebianDog (yet it works okay on Fred's openbox version). Oddly, if I install the official Debian mplayer it only uses around 25% cpu, which is reasonably acceptable especially since mesa acceleration isn't working on the machine anyway. I'm pretty sure part of it isn't mplayer itself so much as some of the other libs that are dependencies in the official Debian version. I do note that the openbox distribution has many possibly relevant extra libs installed by default in the main squashed filesystem. I don't know which to try in jwm debiandog, but I'm also concerned that unless I remaster the filesystem each time when I try to add any libs then there may be timing/stability issues (??) due to the extra layers involved and heavy xz compression. Am I the only one having these issues?

In summary, personally, I'd rather we tested with gz compression, at least at the early beta stage, to see if that provides better stability. Otherwise it is hard to know if weird issues are to do with extra libs being needed or are the result of xz decompression speed and load in conjunction with the multiple union layers. Aside from any advantages for Copy to Ram, I think DebianDog could afford to be considerably larger (up to 200MB) especially considering it is a Dog not a Puppy at the end of the day and most Puppy isos these days are up around 160MB or so.

Stability along with efficiency are the most important factors for me in the end. If that improves with gz rather than xz, then that is what I'd prefer. I certainly like an iso under 200MB, but less than that makes no difference of much relevance to me.

To be frank, I don't think, for most things at this stage, Toni's old 128MB RAM clunker, with large swap partition and low CPU speed, is a good test machine. That machine is going to have performance issues simply because of the amount of swapping it is going to do...!! But machines like my 'old' Pentium M, 1.6GHz, 1 GB RAM should be very capable (though not brilliant) at running current stable Wheezy though they may be struggling with heavy xz compression. Faster newer machines probably work fast enough anyway, such that performance (or iso size) is not really important for them.

[stemsee]

Saintless

Is there any chance some of the modules were compiled for another kernel? I have swapped kernels and modules around and ended up with mismatching mods on puppy precise resulting in exactly the same situation i now experience on debiandog, ie existing firmware/mods not loading. just a thought.

Otherwise i shall wait for your distro to mature before settling on it.
Cheers

ps
Have you thought about joining woof-ce development on github?

[saintless]

Hi, William.

DebianDog is a base to make what ever you like from it. For best performance use full install on ext partition. If you like gzip compression just extract the module and make new one with mksquashfs only command.
Or edit /opt/bin/remasterdog by removing the xz compression from mksquashfs command and remaster this way all the time.

All modules on the site are also xz compressed. Just for information we use over 80% of the space we have from Smokey already. Consider also this as point to keep the modules small as possible.

DebianDog is not something much different from Debian Live CD. It is Debian Live CD with different WM and scripts to use sfs files. I keep saying this again and again. There are plenty versions of Debian Live CD to be tested for mplayer performance for example. If DebianDog has some problem with mplayer CPU usage the first base to compare is not Puppy but Official Debian Live CD. And not with porteus-boot but with official wheezy initrd.img (live-boot-3).
Otherwise it is similar to test and compare Slackware and Ubuntu mplayer performance.

DebianDog is not new invented OS. Any improvements we can add are welcome as long it does not change Debian structure and Debian behaviour inside the main module and official initrd.img for live-boot-3.

Toni

[mcewanw]

I was planning on trying a full install and also resquashing to gz by default in my tests. Also thought about trying similar distributions like Crunchbang as a comparison. If you aware of any particularly nice/small/functional Debian Live based distributions to try let me know though of course I google around anyway. Thanks, William

Enough computing for me today though. The sun is out and missing the kayak.

EDIT: @stemsee. You haven't explained in much useful detail what "situation" you "now experience on debiandog" so I don't see how we could help without that. For example, what firmware/modules aren't loading? I perhaps should clarify that I am myself experiencing no such issues. No one if forced to use DebianDog, which is just another community developed distribution - in this case using multi-user capable Debian Live rather than a single-user woofed Puppy. This thread is just about trying to improve DebianDog as best we can as a community. Being the developers' thread, I was just stating my opinion that I would prefer gz to xz compression at the beta testing stage (which as Toni reminds me I can arrange myself anyway). Not that I always agree with Toni! :-)

I simply want to eliminate any factor that could cause misinterpretation of test results. On the whole DebianDog is working just as well as any Linux distribution I have tried, and more flexibly than most. Being purposively as small as reasonably possible it doesn't come with the likes of mesa graphics support, but that can be fetched with apt-get in a matter of minutes - the only issue is that my hardware is too old to use mesa acceleration without a recompile, but that is the same for any modern Linux. I need to use an older 2.6.x kernel otherwise, such as that provided on GuyDog but that doesn't meet my multi-user/server/desktop needs, which DebianDog does.

Having said that, I am a fan of many of the Puppies produced by pemasu, and also of GuyDog by Iguleder, which I use regularly because it's mesa graphics drivers are perfect for my old machine here. I am not so fond of precise - it just doesn't work so well on my machines, but I do like old Slacko 533, which was a great Puppy release.

[saintless]

Hi, William.

This is DebianDog base (no WM):
http://live.debian.net/files/stable/ima ... andard.iso
You will find KDE, XFCE, LXDE, Gnome and rescue cd version there:
http://live.debian.net/files/stable/ima ... so-hybrid/

I will see if I can make older 2.6.x kernel to work in DebianDog as separate module. But it will work only with gz compressed main module since xz is not supported in 2.6.x

Toni

[saintless]

Hi, Stemsee.

Is there any chance some of the modules were compiled for another kernel?

Very possible if I had something to do with this compiling. But since it is official kernel and firmware included in official Debian Live Cd and available for download ready for use from Debian repository - the answer is no.

Otherwise i shall wait for your distro to mature before settling on it.

I think it will be best. It may take some time till it mature as Puppy did...

Have you thought about joining woof-ce development on github?

My view of Puppy linux future development is much different from what happens there. I prefer to explore the option to make Debian Puppy lookalike.

Toni

[mcewanw]

I will see if I can make older 2.6.x kernel to work in DebianDog as separate module. But it will work only with gz compressed main module since xz is not supported in 2.6.x

Toni

That's nice of you Toni. Be interesting to see how it performed. I was considering trying something like that with the kernel from GuyDog, since then I could take the mesa drivers out of there too.

Code: Select all

GuyDog:
uname -a
Linux puppypc7880 2.6.39 #1 SMP PREEMPT Fri Aug 5 11:19:11 GMT-2 2011 i686 GNU/Linux

However, being unfamiliar with the process I'd probably never get round to it because it would take me too long. I presume these kernel modules comprise the separate vmlinuz along with a squashed fs containing the compatible modules and firmware basically?

The package manager in GuyDog accesses squeeze packages, though I imagine Iguleder compiled the kernel himself. Still I imagine you are thinking of trying a kernel from squeeze distribution, which I know you are familiar with since you started off with a Light Debian Core Squeeze. Of course a 2.6.x version might perform better on your older machine too.

Myself and my two year old son are clearly back from our two hour kayak trip now. It is Autumn here now and the sea (and moreso the river estuary) was like refreshingly cold tap water, but the sun was still reasonably nice and warm at around 19 deg C, with no wind and just relaxing rolling waves. No kayak tomorrow though, we'll be cruising around on a bicycle instead.

[saintless]

Uploaded new ffmpeg2sfs with information button added instead separate readme.txt file. (Thank's, Fred!)
http://murga-linux.com/puppy/viewtopic. ... 385#773385

Fred,
can you suggest some better way or command for resizing save file which can make it also smaller? Check the attached archive. I use resize2fs but it breaks something in the save file structure if it is in use while resizing. fsck finds many errors after that. Especially if I choose to shrink it while it is in use. It will be nice to make it smaller but may be it is not safe enough at least with resize2fs.

Toni

[saintless]

Hi, William.

Unfortunately I still struggle to make Puppy kernel with Puppy initrd.gz file to work with DebianDog main module. I got it to boot but it errors starting X and drops me back to command prompt. Still trying because this can give us new type of save file options.

Yes, I will make separate module from Squeeze. I will check out what available kernels are included in Squeeze Live Cd.

Toni

[fredx181]

can you suggest some better way or command for resizing save file which can make it also smaller?

I'm almost sure that the only reason for the problem is that you resize when savefile is mounted.(It's just not possible AFAIK)
When unmounted it's always ok (just did a few tests).
But anyway I would add a e2fsck line in the script before and after resize command so becomes this:

Code: Select all

e2fsck -y $P/$F
echo y | resize2fs -f $P/$F ${Size}m		# format Image file
e2fsck -y $P/$F

The moment I complained about a "not clear head" was the start of being ill the last couple of days, mostly lying in bed.
So until I'm a little better there won't be much activity from me here.

Fred

[saintless]

Thank you, Fred!
Take care of your health and rest. Nothing important needed anyway. I just try to modify some scripts from you and Terry for different actions we need.

Unmounted save file resizing works fine. Regarding man page resize2fs: http://linux.die.net/man/8/resize2fs

If the filesystem is mounted, it can be used to expand the size of the mounted filesystem, assuming the kernel supports on-line resizing.

But I need to test proper if it works with ext2, ext3 and ext4 the same way. The problem is shrinking mounted save file. resize2fs can't do it safe.
I just want to explore if we can use resize2fs for this or to search for another option.

Toni

[saintless]

Hi, William.

This is just for your testing and I will remove it from the site after you confirm download and boot without problems.
Unsquash the main module and make gz compression.

Code: Select all

unsquashfs -d /live/image/work-folder /live/image/live/01-filesystem.squashfs

Code: Select all

mksquashfs /live/image/work-folder /live/image/live/01-gzip.squashfs

Download and extract the archive in /live (it is 133Mb with 3 kernels included):
http://smokey01.com/saintless/Fredx181/ ... -2.6.x.zip
There is readme text inside.

Code: Select all

Use live-boot-2 code only and live-rw save file if needed:

For kernel-2.6.32-5-486
vmlinuz1a
initrd1a.img

For kernel-2.6.32-5-686
vmlinuz2a
initrd2a.img

For kernel-2.6.32-5-686-bigmem
vmlinuz3a
initrd3a.img

If you do not use live-rw save file on your boot partition it will be mounted read-only. To remount it rw just single click /opt/bin/remount-rw script.
If some of the kernels worth permanent upload I will make separate module with update dpkg option later. This one is only for testing.

Toni

[fredx181]

Hi Toni
I saw in the other thread about problems with the firefox-portable deb.
Sylvander is right: some things are wrong, here's a good one.
Fixed menu-update (runs update-menus in postinstall script)
Fixed ability for default_web-browser when changing in "set-default-apps" (didn't previously work because of symlinked to portable script /opt/bin/firefox , which I changed so that it works now.)

https://drive.google.com/file/d/0ByBgCD ... sp=sharing

Fred

[saintless]

Thanks, Fred.
I will replace it tomorrow morning.
If you can help there with porteus-boot from CD and searching for changes.dat file on top of any hard drive partition it will be nice. I'm not sure what is the right code for this but I will do some testing tomorrow.

Toni