Light-Debian-Core-Live-CD-Wheezy + Porteus-Wheezy

[fredx181]

About apt2sfs - sorry for not explaining better. The point is to be able to update dpkg with information what is installed inside the module created with apt2sfs. We do not need included in status, available and info folder the programs installed inside save file from the user.
.....
.....

Thanks for the explanation Toni, I'll see what I can do in the next days and ask in case I am stuck.
Btw, You might have been right when you were joking a while back to be careful apt2sfs script being smarter than me

Fred

[saintless]

Thanks, Fred

Apt2sfs change is not very important. If it is not easy just leave it as it is.

Remastercow added to fixes post. The checkbox deactivated by default and changed checkbox title. I don't think it is clear enough but maybe we will think of something better till next iso update.
RemasterCow-last - download


Toni

[fredx181]

Hi Toni,

I don't think it is clear enough but maybe we will think of something better till next iso update.

Yes, I also think it needs small change, we'll see later.
I (hopefully) understood what's needed for your idea about apt2sfs, here's modified one: Attached apt2sfs.tar.gz
- var/lib/dpkg/infonew includes the files only from installed packages by apt2sfs.
- var/lib/dpkg/statusnew and availablenew has packages info from DD iso + packages installed by apt2sfs (nothing from possible installed or removed packages in savefile).

Fred

[saintless]

I (hopefully) understood what's needed for your idea about apt2sfs, here's modified one: Attached apt2sfs.tar.gz

Great, Fred, thank you!
Quick tested and it does exactly what I need. I can use now modules made with apt2sfs for dpkg synchronize testing. Thank you again for the fast mods!

Toni.

[fredx181]

Quick tested and it does exactly what I need. I can use now modules made with apt2sfs for dpkg synchronize testing.

Ah, nice! The hard part was to know first what exactly was needed, the rest turned out easier than I thought.
Good luck with your savefile-update project, very nice initiative!

Edit: I'm not sure if I mentioned already in the past:
I made change sometime ago in linuxrc that makes it possible (when using porteus-boot) if module loaded that has .wh files inside, it will behave exactly like savefile (record added files as well as deleted files).
Didn't test it much though.

Fred

[saintless]

Hi, Fred.

I don't think it is clear enough but maybe we will think of something better till next iso update.

Yes, I also think it needs small change, we'll see later.

It needs more - separate help button as for dpkg registration. We can't explain this "remove files record" option clear for the user in one line only. Here is what I found so far:

1. Works with live-boot-3x and porteus-boot.
Doesn't work with live-boot-2x and I can't find the reason yet, but the problem is not in RemasterCow script. Using direct command after removing program:

Code: Select all

mksquashfs /live/cow /live/image/live/03-new-test.squashfs

also does not work for live-boot-2x. The uninstalled program is still there after reboot. I guess it is the way initrd1.img was created with downgraded live-boot in Wheezy. I need to test with official Squeeze live cd to confirm. I will do it in the next days.

2. We need to place the new module inside /live and reboot. Load with sfs does not work for removed files.

3. If "remove files record" option is used we need to uncheck dpkg registration box, otherwise /var/lib/dpkg/info will still contain removed files visible (because the name inside the module will be infonew).

Anyway I think it is still good option and improvement for RemasterCow. We just need to explain proper how to use it in separate help window as dpkg registration.

Toni

[saintless]

...Doesn't work with live-boot-2x and I can't find the reason yet, but the problem is not in RemasterCow script. Using direct command after removing program:

Code: Select all

mksquashfs /live/cow /live/image/live/03-new-test.squashfs

also does not work for live-boot-2x. The uninstalled program is still there after reboot. I guess it is the way initrd1.img was created with downgraded live-boot in Wheezy. I need to test with official Squeeze live cd to confirm..

Quick test shows it does not work with Squeeze live cd also. Seems whiteout files is supported only for save file in Squeeze. This could explain why it does not work in live-snapshot also. Live-snapshot is compressed archive and squashfs is also compressed. Save file is uncompressed. I will test some more but I doubt I will be able to fix this issue in live-boot-2x.

Toni

[fredx181]

Hi Toni,

Quick test shows it does not work with Squeeze live cd also. Seems whiteout files is supported only for save file in Squeeze. This could explain why it does not work in live-snapshot also. Live-snapshot is compressed archive and squashfs is also compressed. Save file is uncompressed. I will test some more but I doubt I will be able to fix this issue in live-boot-2x.

I think it doesn't have to do with compressed or not.
It's the way of (re)mounting the module. As I changed line 332 in linuxrc:

Code: Select all

echo "  $m  $NAME"; mount -no remount,add:1:/memory/images/$NAME=rr+wh aufs /union

Added '+wh' which makes it work with deleted files.(standard porteus-boot doesn't support whiteout files for modules also)
Also to experiment you could try change line 148 in loadmodule script (add +wh) to:

Code: Select all

 mount -n -o remount,add:$POSITION:$CHNGS/$BUNDLE=rr+wh aufs /

Works then with removed files loading module on the fly.

Edit: I've done some digging but no testing yet, have to go out for the afternoon now.
You can try to change line 1117 in 'scripts/live' (from initrd1.img) to:

Code: Select all

		roopt="rr+wh"

Fred

[saintless]

Thanks, Fred!

And I thought all we need is to change RemasterCow to include wh. files...

Also to experiment you could try change line 148 in loadmodule script (add +wh) to:

Code: Select all

 mount -n -o remount,add:$POSITION:$CHNGS/$BUNDLE=rr+wh aufs /

Works then with removed files loading module on the fly.

Do you see any possible reason not to make this default for loadmodule in the next version? I think we should make it work with wh. files on the fly.

Edit: I've done some digging but no testing yet, have to go out for the afternoon now.
You can try to change line 1117 in 'scripts/live' (from initrd1.img) to:

Code: Select all

         roopt="rr+wh"

Thanks, I will test it in the next hour and post the result.

Edit: No, it does not work with roopt="rr+wh"
I will experiment more with /scripts/live today.

Edit2: Sorry, Fred, it works changing to roopt="rr+wh"
I just used the packing command in the wrong directory with extracted original initrd the first time.
Now I will have to repack all lb2 initrd files for all separate kernel modules for next iso update again

Toni

[anikin]

Hi Toni, Fred et al,

I think the sfs files with plain text icon are something wrong in rox mime types. If you use Xfe the mounting scripts work with any type sfs/squashfs (old or new version compression). I do not think using filemnt will solve this issue in Rox but maybe I'm wrong. We have workaround for such sfs files with plain text icon and it is Right Click -> SendTo -> mnt-img
We really need someone with Rox setup experience to take a look in DebianDog setup. I added all suggestions and settings posted in the thread but I can't do more. The problem with sfs file with plain text icon was reported from William when he found ffmpeg2sfs creates sfs with plain text icon in Rox few iso updates back. But the same files with plain text icon are correctly shown and single click mounted in Xfe.

Although, not someone with experience, I did take a look, but not at DD, rather Puppy Raring-5694 setup. And saw the following: there's a directory, named "Choices" in Puppy /root home. I believe it belongs to Rox, as it contains MIME-types, among other things and this directory is missing in DD. Can this explain the mounting mystery and plain text icons of some squashfs files in jwm/rox DD? Toni, if flmnt works well for you in Xfe as it does for Fred in Thunar, can you make it The only and default tool and remove all the other variants from DD to avoid further confusion? Also, will it be it wise to not rename flmnt to mnt-img, keeping the original name as in Puppy and placing it in usr/bin? BTW, flmnt plays really nice here. Thank you, Fred for the deb.

[saintless]

Hi, Anikin.

Although, not someone with experience, I did take a look, but not at DD, rather Puppy Raring-5694 setup. And saw the following: there's a directory, named "Choices" in Puppy /root home. I believe it belongs to Rox, as it contains MIME-types, among other things and this directory is missing in DD. Can this explain the mounting mystery and plain text icons of some squashfs files in jwm/rox DD?

It is possible, but installing rox with apt-get does not create such directory in /root. I remeber Terry was trying to fix Rox mimetypes automatically and he had troubles. I remeber he wrote something about extra package has to be installed but the size was too much to add at the moment. All files in Rox are set by right click - Set run action. I do not want to touch Rox setup because it is very easy to break what is working now for user and root. Till someone provides step by step instruction how to setup better Rox in DebianDog without breaking what works now it stays as it is.

Toni, if flmnt works well for you in Xfe as it does for Fred in Thunar, can you make it The only and default tool and remove all the other variants from DD to avoid further confusion?

if it works well with XFE (quick test shows it does) - yes. But I will keep the other scripts in /opt/bin/special/old. There will not be confusion since the scripts are not in PATH.

Also, will it be it wise to not rename flmnt to mnt-img, keeping the original name as in Puppy and placing it in usr/bin?

What we can do safe without need to set manually Rox and Xfe file associations and risking to break or miss something is to have /usr/bin/filemnt with links to /opt/mnt-img or /usr/local/bin/default_fs-image.rox. I will check what is better and after testing will be added to the changes post.

Toni

[fredx181]

Hi Toni,

Do you see any possible reason not to make this default for loadmodule in the next version? I think we should make it work with wh. files on the fly.

I think it can't do any harm, the user can decide to include .wh files or not in module (with modified RemasterCow)

Edit2: Sorry, Fred, it works changing to roopt="rr+wh" Smile

Haha, successful first shot

Now I will have to repack all lb2 initrd files for all separate kernel modules for next iso update again Smile

That's the consequence I'm afraid, can you add also the new initrd1.xz with systemd fix to it?

Fred

[anikin]

It's all old news. Choices was renamed to .config. So, technically nothing is missing, as the contents of both directories are identical. Maybe, Puppy uses an older, or patched rox version. Anyway, some useful reading for retarded Rox/JWM fans, like myself http://rox.sourceforge.net/desktop/book ... l/163.html

[saintless]

That's the consequence I'm afraid, can you add also the new initrd1.xz with systemd fix to it?

Sure, Fred, I will replace last linuxrc in every available porteus initrd file inside separate kernel modules in the next days.

Toni

[fredx181]

Sure, Fred, I will replace last linuxrc in every available porteus initrd file inside separate kernel modules in the next days.

Sorry, after second thought, I was probably thinking to easy. I thought at first it was only a matter of replacing initrd1.xz into all the kernel modules archives, but there's more to it, thanks!

Fred

[saintless]

No problem, Fred
I will use separate computer for rebuilding only kernel modules one of the next days.

Just in case you like to test sometimes the combination of restdpkg and restore-dpkg-status attached synchronize-dpkg.zip archive - (early base and testing version). In general it does what I want (at least I hope so). BTW while testing found one more fix for Jwm version for moded roxterm-gtk2 deb package.

Example testing situation (any different test is welcome):
Created save file with older version of DebianDog (no Roxterm-gtk2 installed there for example). Removed Synaptic and installed some new programs, but Synaptic has dependency needed for Roxterm. This dependency is auto-removed because roxterm is not installed. The available and status files inside this save file do not include information about roxterm and others later added in DD packages.
Then I boot latest DebianDog (Jwm or openBox) loading the same save file made with previous version. Then I load 4 different sfs files made with RemasterCow (no-dpkg-registration) and with latest apt2sfs from you. One of the modules is gawk.
If I ran apt-get update and apt-get -f install I get no errors and I can even install gawk, but starting roxterm fails because of missing dependency and roxterm is not installed according current dpkg status as few more included in later version packages.
Then I run synchronize-dpkg and at the end status is updated with correct information from the save file, main 01-filesystem.squashfs module and the other 4 sfs modules loaded at the moment.

What the script needs (and exits with error if it is not found):

1. Gawk installed or loaded from sfs.
2. The boot folder name must be live and the main module name 01-filesystem.squashfs.
3. For live-boot it needs showmounts parameter.

What the script does:
1. Search for /var/lib/dpkg/available inside main 01-filesystem.squashfs
2. Search for /var/lib/dpkg/infonew and availablenew in any extra loaded squashfs modules.
3. Merge all found and remove duplicate entries creating new /var/lib/dpkg/available file.
4. Copy the content of every /var/lib/dpkg/infonew in /var/lib/dpkg/info
5. Reads /var/lib/dpkg/info and /var/lib/dpkg/available and creates new /var/lib/dpkg/status
6. Checks and reinstalls (if found) roxterm-gtk2 and x11-xserver-utils - this is mostly Jwm version menu entries fix. OpenBox has no roxterm and it will not be installed.
7. Runs apt-get update and shows message to run apt-get -f install
After fixing few missing dependencies dpkg is happy.

What I like to add in next updated version is script with options to run restdpkg (if only sfs modules are loaded), restore-dpkg-status (when status file is deleted by mistake or broken somehow), synchronize-dpkg (when the save file is not made on the same DebianDog version and for any other possible dpkg problem).
I'm not 100% sure synchronize-dpkg does its job proper yet. More testing needed to confirm.

Toni

[fredx181]

Hi Toni,

Just in case you like to test sometimes the combination of restdpkg and restore-dpkg-status attached synchronize-dpkg.zip archive - (early base and testing version).

Thanks Toni, looks promising!
I tried already once (using savefile made with older DD) but it was not the right testcase because everything was ok already before running your script.
So I need to find different but similar situation as you described about roxterm/synaptic.
I'll probably get back later today.

Edit: Having positive results from very simple test:
On openbox beta version (without sfs-get_smokey-get) I removed wget (dependency of sfs-get_smokey-get) and installed also some programs.
Then booted latest DD with the same savefile , tried apt-get -f install, that did nothing, ran your script and did again apt-get -f install and wget got installed.

Fred

[saintless]

Thanks, Fred, I like to read positive results

I like to test it in more complicated situations as save file created from other Debian Wheezy Live CD (official and based on Debian Wheezy) and using the same save file with DebianDog.
I think live-boot-3x function needs some more work in case mixed load of extra squashfs modules on boot (inside /live) and after boot (with load modules) - in both cases the mount point is diiferent with live-boot-3x. I have possible solution for this without need to change loadmodules and I will test it later. But even if I can't - restdpkg covers this extra loaded modules dpkg-update for all boot methods. Also some more moded from official Debian packages could be included for reinstall to prevent adding unneeded dependencies.
Then I will add extra script that calls restdpkg, restore-dpkg-status and synchronize-dpkg in different situation and make deb package with gawk and wget added as dependencies. Then we will deside to keep it as extra tool for download when it is needed or include it in next update.

Toni

[fredx181]

Hi Toni,

Inspired by your synchronize script I thought it might be good to have some sort of simple DebianDog repository. So made it easy for you to create one on http://www.smokey01.com/saintless/Debia ... /Included/
But maybe you have reason not to want that.

If you want can you test by doing the following?:
I prepared just for testing this 'Included' directory.
Extract the attached included.tar.gz, then in the folder 'included' are the files: Packages, Packages.gz, Contents, Release etc..
Upload all these to http://www.smokey01.com/saintless/Debia ... /Included/

Then to make the repository available:

Code: Select all

# Get the signed key
cd /etc/apt/trusted.gpg.d/
wget "http://www.smokey01.com/saintless/DebianDog/Packages/Included/debdog.gpg"
# Add to sources
echo 'deb http://www.smokey01.com/saintless/DebianDog/Packages/Included/ ./' >> /etc/apt/sources.list
apt-get update

There are a few packages not included in the 'Packages' list because they gave errors when updating the package lists (I tested first with creating local repo).
The reason is that these packages have nothing after 'Depends:' in control file.
So they should be changed to at least something like: 'Depends: libc6' I think.

Edit: Re-uploaded included.tar.gz with all packages included now (Toni fixed the 'Depends')

Fred

[saintless]

Thanks, Fred

I don't have time to test this repository proper today but all files are uploaded and the packages without dependencies replaced.
I do not mind having repository. My only objection is if we need to maintain this repository often by changing packages list content or other files every time new package is uploaded. I suggest if this is the case to do it after every 10 new uploaded packages or something similar.
I'm not sure if some packages with the same name inside control file will make a problem or that some uploaded files are not deb packages?
What is your idea regarding synchronize-dpkg script? To auto-add this repository in sources.list or something different?

Toni