Light-Debian-Core-Live-CD-Wheezy + Porteus-Wheezy

[mcewanw]

Yes, these examples work for live-boot but it should be noted that in case a user wants to switch to porteus-boot: When changing name of "live" folder, porteus-boot doesn't work anymore.

Yes, just for live-boot Fred. But I'm surprised the Porteus-boot from= code isn't used to point to where the filesystem squashfile is. The changes= part does seem to logically point to where, for example, a changes directory would be. I'm thinking there is a mistake in the from= code. Was that all from Porteus or have you modified it somehow. If you modified it, perhaps it is possible to make it point to filesystem squash file whether it is in live folder or not, which would be more logical to me? I remember reading that, as things stand, you are able to just put from=/ (which again seems a bit illogical).

In other words, why is the from= in porteus-boot assuming 'live' rather than usefully being used to specify the full path to 01-filesystem.squashfs (i.e. from=/live/ or if you have 01-filesystem.squashfs in /portdebdog, from=/portdebdog/)?

[fredx181]

Yes, just for live-boot Fred. But I'm surprised the Porteus-boot from= code isn't used to point to where the filesystem squashfile is. The changes= part does seem to logically point to where, for example, a changes directory would be. I'm thinking there is a mistake in the from= code. Was that all from Porteus or have you modified it somehow. If you modified it, perhaps it is possible to make it point to filesystem squash file whether it is in live folder or not, which would be more logical to me? I remember reading that, as things stand, you are able to just put from=/ (which again seems a bit illogical).

Original Porteus has the "porteus" folder with folders 'base', 'modules' etc.. inside.
It's the variable "FOLDER" in the initscript.
For Porteus-Wheezy I changed that variable to "debian" to make distinction with original Porteus.
Then later changed to "live" to have it the same default name as live-boot.
AFAIK, that variable is needed, a whole lot of other things depend on it.
But maybe it's possible what you suggest (to point to filesystem squash file) but it probably needs changing a lot.
"from=/" works only when the live folder is at the root of the partition.(in fact "from" is parent directory of "live")
Btw, from= can be left out when live is on root of partition.

Fred

[saintless]

Hi, William.
Here is the post from Fred:
http://murga-linux.com/puppy/viewtopic. ... 797#769797
It was a choice for using both boot methods without symlink in /live/base for Fat partition.
The changes are in porteus initrd1.xz file and are special for DebianDog now. Different from standard porteus boot code.
It was a choice to change porteus-initrd or eventually always to be forced to use live-media-path= for live-boot.
Some boot options for porteus don't work anyway with DebianDog. No use to try changing porteus initrd more. It is involved with a lot of testing which we do not have at the moment.
Also I do not like the idea to rebuild again all initrd.xz files included inside separate kernel modules.

Toni

[mcewanw]

in fact "from" is parent directory of "live"

Putting it that way does provide a logical understanding of from=. It's just a minor pity that a fixed folder named 'live' is required for porteus-boot case only, but as Toni and yourself say, trying to change that would involve a lot of work and not worth the effort really.

By the way, I like that conkytoggle in your openbox version Fred! Simple, but nice to have.

[saintless]

Hi, Fred.
can you, please, make final edit if needed and we can upload new version on the site.
/root changed to $HOME
added information for dpkg registration.

Code: Select all

rm -rf "/mnt/$DRV/$WRKDIR"$HOME/.fltk
rm -f "/mnt/$DRV/$WRKDIR"$HOME/.local/share/recently-used.xbel
rm -rf "/mnt/$DRV/$WRKDIR"$HOME/.local/share/Trash

Toni

[saintless]

Hi, all.
Nothing more from me to add in Utilities and HowTo threads. Anything new you can find or suggest or questions if something is not clear enough just post it there as next post information.
I will focus most on testing DebianDog from now.

Toni

[fredx181]

Hi Toni,
Thank you!!!
Very clear explanation.
It's nice to have it this way with the choice for dpkg registration or not and the info about it, we should cooperate more often
The only thing I changed now is the size of the window so it shows all info.

rm -rf "/mnt/$DRV/$WRKDIR"$HOME/.fltk
rm -f "/mnt/$DRV/$WRKDIR"$HOME/.local/share/recently-used.xbel
rm -rf "/mnt/$DRV/$WRKDIR"$HOME/.local/share/Trash

Good additions, I first thought it wouldn't work for puppy user but it does.
It seems that the ktsuss program does that because when using "su - root" the outcome of 'echo $HOME' is /root.

There is one more option here which is the best one - some script for the future that will auto separate the packages from /var/lib/dpkg/info in new status and available and auto-make update-script.

Could you explain a little more about what that magic script exactly should do? It's a bit confusing for me.
I've been searching on the net for a script that reads from /var/lib/dpkg/info and transform in a status file but thats more for when dpkg is broken because of status file missing.

Edit: Just found this one here:
http://tuxx-home.at/archives/2005/02/16/T13_36_14/
It seems to work well and the path variable can be changed to live/cow/..... so it reads from the /live/cow/var/lib/dpkg/info (just guessing at the moment, not sure if this could be useful).
It needs gawk installed,btw.

Fred

[saintless]

There is one more option here which is the best one - some script for the future that will auto separate the packages from /var/lib/dpkg/info in new status and available and auto-make update-script.

Could you explain a little more about what that magic script exactly should do? It's a bit confusing for me.
I've been searching on the net for a script that reads from /var/lib/dpkg/info and transform in a status file but thats more for when dpkg is broken because of status file missing.

Hi, Fred.

Just finished this today. You will find the basic idea in this post:
http://ns1.murga-projects.com/puppy/vie ... 1e5#774461
But automating this is very complicated. I see automate script to do the following actions inside the separate module before making new squashfs:

1. Read the package name inside /var/lib/dpkg/info from package-name.list (skipping .list and package-name as name to search).
2. Find the package name and mark the package section information (maybe impossible to be done because the section is different number of lines for every package) inside status and available files.
3. Create status-(name of the new created module) and available-(name of the new created module).
4. Copy the marked sections form status and available inside status-(name of the new created module) and available-(name of the new created module) and leave one empty line between the packages and 2 empty lines at the end of the file.
5. Delete status and available from the working dir.
6. Create /opt/bin/(name-of-module)-executable-script
7. Add inside this script:

Code: Select all

#!/bin/bash

cd /var/lib/dpkg
cp -f status status-old
cat status-(name of the new created module) >> status
rm status-(name of the new created module)
cp -f available available-old
cat available-(name of the new created module) >> available
rm available-(name of the new created module)
cd /opt/bin
rm -(name of module)-executable-script 

Not sure at all if it can be done to work well. This is only idea to think about for the future.

Toni

Edit: Yes, this looks like something what we need:
http://tuxx-home.at/archives/2005/02/16/T13_36_14/
I have to test this proper but I think only updating status will be enough. available contains information for all installed on the system packages ever. many of them are removed but the information in available stays. The important one is status.

[fredx181]

Hi Toni,

Just finished this today. You will find the basic idea in this post:
http://ns1.murga-projects.com/puppy/vie ... 1e5#774461
But automating this is very complicated.
.......
.......

I knew it would be complicated, but it's soo... complicated!
I need a much more clear head (than I have now) to understand.
Will read more later.
Just a simple question:
What's the goal?
I think it's something like: Every module could have it's own "place" in the system and there's also no space wasted with duplicate libraries or binaries. Hope some of this is right, so then at least I understand something
Tell me if you think I can help with some bash scripting or something.

Fred

[saintless]

Hi, Fred.

It is not clear enough for me as well yet
What we gain with this is to have the same option for every module as every separate kernel module has - option to update dpkg if the user decide to remaster the system instead using always this module separate. In the case with changing kernel it is very important option but not so important for every package.
I also see some risk to give this option with RemasterCow because if some of the packages included inside this separate module are already installed as dependency for another package, then updating status file will put 2 entries for one package and this will break dpkg. This means to make the script much more complicated by adding one more step - searching in status file if some package is already installed (in case the module is created on fresh DebianDog and the user wants to add it on later remastered DebianDog). I will stop here
As I said it is not clear enough for me also yet. I just include some stuff I don't want to forget in HowTo thread for later use.

Do not think about this. Having disable dpkg registration default option is fine and safe method for creating module from changes. I'm happy with that.

BTW I need to fix blkid command to be included in the path when using sudo and su for puppy account in jwm-icewm version. I will add it in the first page fixes.

I will upload new RemasterCow tomorrow morning with pictures and post link to it in the beta thread.

Toni

[saintless]

Hi, Fred. Just for info.

Edit: Just found this one here:
http://tuxx-home.at/archives/2005/02/16/T13_36_14/
It seems to work well and the path variable can be changed to live/cow/..... so it reads from the /live/cow/var/lib/dpkg/info (just guessing at the moment, not sure if this could be useful).
It needs gawk installed,btw.

Thanks to this guy the most difficult part is solved. I will test the script proper to see how it works. It will be nice to include it as restore broken dpkg database option.
I will try to make separate script executing the above actions I wrote in extracted module to see how it goes. But I think separate tool for this (extract ready module and pack it back) will be better instead including all this in RemasterCow.

Toni

[mcewanw]

Yes :) exactly the same for me, it's not so damn slow to compress and btw, with it the system runs smoother, boots up faster, I'm beginning to wonder why we use xz compression!
For that few MB's less size? (yes, I know, more then a few, but AFAIK that's the only real advantage of xz )

Hi, Fred.
The only advantage of XZ is the smaller size and we pay for it with a little bit slower boot, but I doubt many users will experience slower boot or any difference from gzip compression. For Example William did not notice any difference except growing up the module size. I guess it depends on the hardware.
For JWM version the size difference is 32Mb. It counts for Copy to RAM option.

I didn't notice a difference in my one test of gz versus xz when it came to mplayer performance (basically monitoring its cpu load). However, I am concerned in case xz compression might be causing stability or timing problems on some machines, such as mine. It may be fine on fast enough machines, but I've been experiencing some very odd effects with CPU load occasionally being 100% on DebianDog and I have really struggled trying to find out why I'm generally experiencing too high a CPU load with installed mplayer on jwm DebianDog. It is using near 50% cpu on both my machines, which is far too much and restricting what other apps can be running at the same time. Same poor result with Fred's gnome-mplayer when I try it with jwm DebianDog (yet it works okay on Fred's openbox version). Oddly, if I install the official Debian mplayer it only uses around 25% cpu, which is reasonably acceptable especially since mesa acceleration isn't working on the machine anyway. I'm pretty sure part of it isn't mplayer itself so much as some of the other libs that are dependencies in the official Debian version. I do note that the openbox distribution has many possibly relevant extra libs installed by default in the main squashed filesystem. I don't know which to try in jwm debiandog, but I'm also concerned that unless I remaster the filesystem each time when I try to add any libs then there may be timing/stability issues (??) due to the extra layers involved and heavy xz compression. Am I the only one having these issues?

In summary, personally, I'd rather we tested with gz compression, at least at the early beta stage, to see if that provides better stability. Otherwise it is hard to know if weird issues are to do with extra libs being needed or are the result of xz decompression speed and load in conjunction with the multiple union layers. Aside from any advantages for Copy to Ram, I think DebianDog could afford to be considerably larger (up to 200MB) especially considering it is a Dog not a Puppy at the end of the day and most Puppy isos these days are up around 160MB or so.

Stability along with efficiency are the most important factors for me in the end. If that improves with gz rather than xz, then that is what I'd prefer. I certainly like an iso under 200MB, but less than that makes no difference of much relevance to me.

To be frank, I don't think, for most things at this stage, Toni's old 128MB RAM clunker, with large swap partition and low CPU speed, is a good test machine. That machine is going to have performance issues simply because of the amount of swapping it is going to do...!! But machines like my 'old' Pentium M, 1.6GHz, 1 GB RAM should be very capable (though not brilliant) at running current stable Wheezy though they may be struggling with heavy xz compression. Faster newer machines probably work fast enough anyway, such that performance (or iso size) is not really important for them.

[stemsee]

Saintless

Is there any chance some of the modules were compiled for another kernel? I have swapped kernels and modules around and ended up with mismatching mods on puppy precise resulting in exactly the same situation i now experience on debiandog, ie existing firmware/mods not loading. just a thought.

Otherwise i shall wait for your distro to mature before settling on it.
Cheers

ps
Have you thought about joining woof-ce development on github?

[saintless]

Hi, William.

DebianDog is a base to make what ever you like from it. For best performance use full install on ext partition. If you like gzip compression just extract the module and make new one with mksquashfs only command.
Or edit /opt/bin/remasterdog by removing the xz compression from mksquashfs command and remaster this way all the time.

All modules on the site are also xz compressed. Just for information we use over 80% of the space we have from Smokey already. Consider also this as point to keep the modules small as possible.

DebianDog is not something much different from Debian Live CD. It is Debian Live CD with different WM and scripts to use sfs files. I keep saying this again and again. There are plenty versions of Debian Live CD to be tested for mplayer performance for example. If DebianDog has some problem with mplayer CPU usage the first base to compare is not Puppy but Official Debian Live CD. And not with porteus-boot but with official wheezy initrd.img (live-boot-3).
Otherwise it is similar to test and compare Slackware and Ubuntu mplayer performance.

DebianDog is not new invented OS. Any improvements we can add are welcome as long it does not change Debian structure and Debian behaviour inside the main module and official initrd.img for live-boot-3.

Toni

[mcewanw]

I was planning on trying a full install and also resquashing to gz by default in my tests. Also thought about trying similar distributions like Crunchbang as a comparison. If you aware of any particularly nice/small/functional Debian Live based distributions to try let me know though of course I google around anyway. Thanks, William

Enough computing for me today though. The sun is out and missing the kayak.

EDIT: @stemsee. You haven't explained in much useful detail what "situation" you "now experience on debiandog" so I don't see how we could help without that. For example, what firmware/modules aren't loading? I perhaps should clarify that I am myself experiencing no such issues. No one if forced to use DebianDog, which is just another community developed distribution - in this case using multi-user capable Debian Live rather than a single-user woofed Puppy. This thread is just about trying to improve DebianDog as best we can as a community. Being the developers' thread, I was just stating my opinion that I would prefer gz to xz compression at the beta testing stage (which as Toni reminds me I can arrange myself anyway). Not that I always agree with Toni! :-)

I simply want to eliminate any factor that could cause misinterpretation of test results. On the whole DebianDog is working just as well as any Linux distribution I have tried, and more flexibly than most. Being purposively as small as reasonably possible it doesn't come with the likes of mesa graphics support, but that can be fetched with apt-get in a matter of minutes - the only issue is that my hardware is too old to use mesa acceleration without a recompile, but that is the same for any modern Linux. I need to use an older 2.6.x kernel otherwise, such as that provided on GuyDog but that doesn't meet my multi-user/server/desktop needs, which DebianDog does.

Having said that, I am a fan of many of the Puppies produced by pemasu, and also of GuyDog by Iguleder, which I use regularly because it's mesa graphics drivers are perfect for my old machine here. I am not so fond of precise - it just doesn't work so well on my machines, but I do like old Slacko 533, which was a great Puppy release.

[saintless]

Hi, William.

This is DebianDog base (no WM):
http://live.debian.net/files/stable/ima ... andard.iso
You will find KDE, XFCE, LXDE, Gnome and rescue cd version there:
http://live.debian.net/files/stable/ima ... so-hybrid/

I will see if I can make older 2.6.x kernel to work in DebianDog as separate module. But it will work only with gz compressed main module since xz is not supported in 2.6.x

Toni

[saintless]

Hi, Stemsee.

Is there any chance some of the modules were compiled for another kernel?

Very possible if I had something to do with this compiling. But since it is official kernel and firmware included in official Debian Live Cd and available for download ready for use from Debian repository - the answer is no.

Otherwise i shall wait for your distro to mature before settling on it.

I think it will be best. It may take some time till it mature as Puppy did...

Have you thought about joining woof-ce development on github?

My view of Puppy linux future development is much different from what happens there. I prefer to explore the option to make Debian Puppy lookalike.

Toni

[mcewanw]

I will see if I can make older 2.6.x kernel to work in DebianDog as separate module. But it will work only with gz compressed main module since xz is not supported in 2.6.x

Toni

That's nice of you Toni. Be interesting to see how it performed. I was considering trying something like that with the kernel from GuyDog, since then I could take the mesa drivers out of there too.

Code: Select all

GuyDog:
uname -a
Linux puppypc7880 2.6.39 #1 SMP PREEMPT Fri Aug 5 11:19:11 GMT-2 2011 i686 GNU/Linux

However, being unfamiliar with the process I'd probably never get round to it because it would take me too long. I presume these kernel modules comprise the separate vmlinuz along with a squashed fs containing the compatible modules and firmware basically?

The package manager in GuyDog accesses squeeze packages, though I imagine Iguleder compiled the kernel himself. Still I imagine you are thinking of trying a kernel from squeeze distribution, which I know you are familiar with since you started off with a Light Debian Core Squeeze. Of course a 2.6.x version might perform better on your older machine too.

Myself and my two year old son are clearly back from our two hour kayak trip now. It is Autumn here now and the sea (and moreso the river estuary) was like refreshingly cold tap water, but the sun was still reasonably nice and warm at around 19 deg C, with no wind and just relaxing rolling waves. No kayak tomorrow though, we'll be cruising around on a bicycle instead.

[saintless]

Uploaded new ffmpeg2sfs with information button added instead separate readme.txt file. (Thank's, Fred!)
http://murga-linux.com/puppy/viewtopic. ... 385#773385

Fred,
can you suggest some better way or command for resizing save file which can make it also smaller? Check the attached archive. I use resize2fs but it breaks something in the save file structure if it is in use while resizing. fsck finds many errors after that. Especially if I choose to shrink it while it is in use. It will be nice to make it smaller but may be it is not safe enough at least with resize2fs.

Toni

[saintless]

Hi, William.

Unfortunately I still struggle to make Puppy kernel with Puppy initrd.gz file to work with DebianDog main module. I got it to boot but it errors starting X and drops me back to command prompt. Still trying because this can give us new type of save file options.

Yes, I will make separate module from Squeeze. I will check out what available kernels are included in Squeeze Live Cd.

Toni