RC7 (STABLE) WeeDogLinux Arch 64 now released

A home for all kinds of Puppy related projects
Message
Author
wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#621 Post by wiak »

I'll add that how to use official void root filesystem post to the user contributions how to list, rufwoof. Its very useful facility.

http://www.murga-linux.com/puppy/viewto ... 29#1029029

Note that once you untar the root filesystem, and rename it firstrib_rootfs, you can simply use the scripts mount_chrootXXX.sh and umount_chrootXXX.sh as a convenience to modify it and add kernel and firmware/modules:

http://murga-linux.com/puppy/viewtopic. ... 57#1028957

As I said in my immediately above post, you can do the same with Void live graphical desktop root filesystem's, such as LXDE, and also with live Slackware (but using Slackware's own kernel/modules/firmware), and others...

http://www.murga-linux.com/puppy/viewto ... 53#1038053

wiak

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#622 Post by rufwoof »

Just a observation ... removing mouse hot corner detection, along with removing xload, cpu temp, caps lock and battery level indicators from my jwm tray ... and the cpu runs with around 5% less usage and 3 degrees lower temperature. Which also means longer running on battery time.
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#623 Post by rufwoof »

Currently I'm using tar to make/restore 'saves' i.e. just simple 'copies' of upper_changes. I've set that the loading (or not) to occur in rdsh3.plug, so the system boots with either upper_changes loaded with the save tarball content, or not. Running 'save' tar's the upper_changes content to a replacement 'save' tarball.

Looking however to move that 'save' loading activity to use rsync instead of tar. rsync is much better for when a small number of changes are being made to a existing large amount of stored changes (compared to having to re-tar the whole).

The more preferable way to run/boot is to use a frugal setup, booting from a usb, loading into ram, such that the usb (boot partition, bootloader, kernel) can be physically detached/isolated once booted. Put out of harms way. For stability a encrypted swap file on hdd helps avoid system lock-up's due to having exhausted available ram. If therefore we're going to have the hdd accessible (swap file) anyway, then we might as well store upper_changes on disk also (rather than in ram as I am currently doing). If that upper_changes is encrypted (encfs) then that makes it secure (laptop lost/stolen, then they can't see the content of our save folder).

As-is the base initramfs system doesn't support encfs nor rsync. Whilst I could compile static versions of those to add to initramfs there's no actual need as we can just mount the main sfs that does (assuming it was built as such) have encfs and rsync. Mounting that however is ro (being a non-layered sfs); However we can also bind mount ... in my case ... /mnt/sda1/VOID6 where the 01firstrib_rootfs and other files/folders are stored, and run rsync (and/or encfs) against files/folders in /mnt/sda1/VOID6 ... such as to load using rsync the upper_changes folder content.

The way encfs works is to have a dot (hidden) folder containing all of the encrypted files/folders, so .upper_changes for instance. That when 'opened' makes the decrypted versions visible in upper_changes.

At least that's the concept, I've yet to actually code/implement that.

Running that way and not only is the MBR/bootloader/kernel physically isolated, but our swap content is encrypted (using a different randomly generated key at each reboot), and so also is the save content. Whilst the system runs mostly in ram, but where swap and save content are disk based, such that we're not limited to ram limits against changes or system loading (system loading limited to disk based swap file size, unlimited (to free disk space limit) amount of 'changes' permissible). Both encrypted swap and save (.upper_changes.rsync) could be stored in a file filesystem, such that that file filesystem could be stored and run from any form of disk format (ext2/3/4/ntfs/fat ...etc.).

Lose the boot usb ... and that just contains OS files anyway, that could just as easily have been downloaded. Lose the laptop/hdd .. and swap, saves are encrypted. If a in-session cracker achieves root access then they can't get to the physically unplugged usb (MBR/bootloader/kernel) but they would have access to the open (decrypted) save content (upper_changes) as well as any HDD data (so as ever backups are mandatory). If the .upper_changes.rsync is also on usb however, then they can't change things and run a 'save' command to make those changes persistent (as the usb is physically detached).

The great thing about wiak's scripts is that making such changes or extensions is so simple. No need to dive into intramfs (cpio extract/change/reform etc.), but instead just simple scripts (rdsh3.plug ...etc.) :)
Last edited by rufwoof on Sun 29 Sep 2019, 23:09, edited 3 times in total.
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#624 Post by rufwoof »

Bug in build_weedog_initramfs05_s203.sh
_rdsh function.

By the time rdsh5 is reached the mountfrom= parameter mount point has been moved from for example /mnt/sda1/VOID6 to under the merged folder (/mnt/layers/merged/mnt/sda1/VOID6). So if you create a rdsh5.plug within /mnt/sda1/VOID6 it isn't 'seen' by _rdsh function and it just drops to a shell instead of sourcing the rdsh5.plug file.

This additional else block within the _rdsh function works for me ...

Code: Select all

# process any grub linux/kernel line rdshN argument to active plugin or debug sh
_rdsh (){
        if `grep -q $1 /proc/cmdline`; then
                # if plugin exists and isn't empty then source it
                if [ -s "${mountfrom}"/${1}.plug ]; then
                        . "${mountfrom}"/${1}.plug
                else # rdsh5 has the mountfrom moved
                  if [ -s /mnt/layers/merged/"${mountfrom}"/${1}.plug ]; then
                        . /mnt/layers/merged/"${mountfrom}"/${1}.plug
                  else
                        # Start a busybox job control debug shell at initramfs/init rdshN code li
                        echo "In initramfs/init at $1. Enter exit to continue boot:"
                        setsid cttyhack sh
                  fi
                fi
        fi
}
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#625 Post by wiak »

rufwoof wrote:Bug in build_weedog_initramfs05_s203.sh
_rdsh function.

By the time rdsh5 is reached the mountfrom= parameter mount point has been moved from for example /mnt/sda1/VOID6 to under the merged folder
It's not a bug rufwoof. I actually commented prior to release of s203 that rdshN.plug would only work up to rdsh4.plug (as the comment in the code says, it will still drop to busybox sh at that point if required). Since I noted you might want a plugin just prior to the chroot I specially put in a pre_switch_root.plug position instead as the relevant s203.sh code shows:

Code: Select all

# If grub kernel-line rdsh5 specified then start busybox job control shell
_rdsh rdsh5

[ "$umount_bootdevice" == "allowed" ] && echo -e "\e[96mYou can now umount bootdevice if you wish\e[0m" >/dev/console

# if pre_switch_root.plug exists in bootfrom directory source it
[ -s merged"${mountfrom}"/pre_switch_root.plug ] && . merged"${mountfrom}"/pre_switch_root.plug
Of course it could be done the way you write it in previous post, but I liked giving actual position name to this plug since it describes the critical position where it is used. Also, don't need additional rdsh kernel line parameter to activate it. I thought of positioning it exactly beside the rdsh5 position, but didn't see any advantage to placing it as near the switch_root as I could whilst /proc and so on still mounted (but only one line further than rdsh5 breakpoint anyway. So just place your save code in pre_switch_root.plug (best anyway, since the other plugin positions are not always fixed as the program develops).

wiak

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#626 Post by rufwoof »

Thanks for the clarity wiak. My brains been like the weather here today (cloudy)
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#627 Post by rockedge »

experimenting on getting a weedog to boot from an ISO in Virtualbox.
not successful yet, getting kernel panic so far

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#628 Post by wiak »

rockedge wrote:experimenting on getting a weedog to boot from an ISO in Virtualbox.
not successful yet, getting kernel panic so far
I'm still to try making any kind of iso. I've never made isos before, since I never use them (though some scripts I've tried in the past have auto-built iso's, just as woof-CE and mklive-stretch) - so I don't know exactly how to do it aside from something to do with isolinux and xorriso.

wiak

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#629 Post by rufwoof »

You might be able to use isomaster, to open up a existing iso from another similar distro, and swap out the vmlinuz, initramfs ...etc. change the .cfg file inside that (like menu.lst ... but different) and then save/write that to another named .iso file.
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#630 Post by rufwoof »

I've been playing with using a small main sfs (voidlinux rootfs as I outlined the other day), and having a massive save file (upper changes) with the rest (X, libreoffice, browser ...etc.).

As a straight folder that's around 3.3GB of upper_changes content. I have that rsync'ing at startup ... so back to a clean copy and mostly rsync runs through quickly (due to just 'differences' being copied across).

Whilst pondering how to better secure that clean copy rsync folder, I decided a reasonable choice is to simply store it in a file filesystem, so its also a single file, and rather than encrypting the whole of that file (I created a 5GB file filesystem that takes a couple of minutes to 'open' (decrypt) when encrypted, I've been experimenting with just encrypting the first 20MB of that file (filesystem), so that when the first 20MB are encrypted it can't be loaded/mounted. Whilst opening it up again is quick, and requires the password to be entered.

The trick is to use dd with the notrunc i.e. with a big.img file of 5GB extract the first 20MB ...

dd if=big.img of=header bs=1M count=20

to a file called header, and then encrypt that smaller 20MB block

openssl enc -AES-256-CBC -in header -out header.encrypted

and then dd that back as the first part of the big file

dd conv=notrunc if=header.encrypted of=big.img

Try and mount that and it complains/fails (obviously). Then to reverse that its pretty much the same but using openssl decode instead ...

dd if=big.img of=header.encrypted bs=1M count=20

then decrypt the header part

openssl enc -AES-256-CBC -d -in header.encrypted -out header

and dd conv=notrunc if=header of=big.img

Mount that and it mounts ok.

I just arbitrarily selected 20MB of 'header' (encrypted) which pretty much flies though both the dd and openssl encryption very quickly. But that could be even smaller, perhaps even just a couple of MB. Doesn't really prevent a concerted cracker effort of changing things as most of the file is open, but does prevent a in session crack potentially making configuration changes and then running the 'save' command to get those changes preserved across reboots. As ever a trade off between speed or security.
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#631 Post by rockedge »

I needed some success after many many attempts to boot the weedog ISO and all resulting in a kernel panic. So I booted a Bionic32-v8 iso in VirtualBox into RAM and used GParted to format the virtual HDD and Grub4Dos to set up the MBR.

Downloaded the weedog scripts created a directory and copied those scripts and a firstrib00-32.plug.

ran those scripts with Bionic32 on VirtualBox and added a menu.lst to boot the new weedog.

I umounted the Bionic32 iso "CD" and booted into a brand new weedog running nicely in VBox.

Then I exported it as a VirtualBox appliance which I tested by creating a new virtual machine and loading and booting the weedog appliance

Which worked great and I feel better now.

The point really is...to be able to watch what a good boot looks like so I can modify the isolinux.cfg correctly to be able to boot weedog burned from a WeeDog ISO...in theory.

But some success is better than none and one step closer :D

Plus fun watching a WeeDog run VBox and WeeDog
Attachments
2019-09-30-184434_640px.png
(156.99 KiB) Downloaded 540 times

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#632 Post by wiak »

I had no idea what I was doing really, but nevertheless I have successfully now booted a WeeDog void from DVD on my second attempt (talk about luck...).

My machine is an old HP elitebook 2530p which happens to come with a DVD writer built into it.

First I read:

https://wiki.debian.org/RepackBootableISO

ignoring all the jigdo stuff (as it said I could...)

So a command for making booting iso using xorriso was there...

My host system was BionicDog64, and I checked and it had isolinux and xorriso installed already.

Then I remembered I had seen possibility to create an iso when remastering with BionicDog's quick-remaster program, so I opened that up to check and sure enough it used xorriso command pretty much the same as in above debian link. The only difference is that above also uses -isohybrid-gpt-basdat, which I might later try since might be how to boot on UEFI machine (but I don't have such a machine here to test just now anyway - my partner has one, but she is away on trip till next week). Anyway, from the above, did the following:

1. created directory weedog_iso_build
2. cp -a /usr/local/isodata/isolinux weedog_iso_build
3. mkdir weedog_iso_build/weelive
4. cd weedog_iso_build/weelive
5. Put weedog build scripts in there and firstrib00.plug_void_default_anyarch and ran:
6. ./build_firstrib_rootfs_103.sh void rolling amd64 firstrib00.plug_void_default_anyarch

followed by (once firstrib_rootfs build complete):

7. ./build_weedog_initramfs05_s203.sh void "-comp lz4 -Xhc"
8. I then deleted all from inside weelive but for initramfs05.gz and 01firstrib_rootfs.sfs and vmlinuz (which I had renamed to simply vmlinuz)
9. Then I cd to just outside weelive and into isolinux folder (i.e. cd ../isolinux) and edited live.cfg to contain:

Code: Select all

label WeeDog
kernel /weelive/vmlinuz
append initrd=/weelive/initramfs05.gz bootfrom=/mnt/sr0/weelive usbwait=12 inram_sz=100% changes=RAM
Really, I was guessing... but ends up working... I guess should also include copy2ram so you can umount and eject the CD or DVD tho I didn't try that.

10 then cd to just outside weedog_iso_build/isolinux (i.e. cd ..) and ran command:

Code: Select all

xorriso -as mkisofs -r -J -joliet-long -l -isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin -partition_offset 16 -V "weedog" -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -o ../"weedog-void-amd64.iso" ../"weedog_iso_build"
The above causes the iso named weedog-void-amd64.iso to be created just outside the weedog_iso_build directory.

11. Then I used pBurn to first blank an ancient old DVD R/W I found in a box I had! And then used pBurn to burn the weedog-void-amb64.iso file onto it (via device /dev/sr0; perhaps /dev/cdrom would have worked - I don't know, I didn't try that). EDIT: just checked and /dev/cdrom is a symlink to /dev/sr0 on my machine so I guess that would have worked as an alternative.

12. And then I rebooted... and to my amazement... it booted up fine.

I hope I have documented the above correctly (I'm just writing it down from memory) but shouldn't be much missing or wrong if anything. Note that I guessed the append argument bootfrom=/mnt/sr0/weelive since it seemed to make sense - anyway, it booted, so seems to be correct for my system anyway; EDIT: but /mnt/cdrom/weelive might have been better - I'm not sure at the moment).

As I said, I did the above whilst running on a BionicDog64 host system; you'd need to make sure isolinux was installed (and xorriso and maybe more) if using a different system (and also maybe check the ISOLINUX directories are in the same place).

Hope the above helps you too rockedge, since same should surely boot directly on a VBox Virtual Machine also. I haven't uploaded the iso anywhere, since it is pretty big (508 MiB) since made no attempt to slim it down (firmware etc) in any way.

EDIT: Note that. to make them easy to find, I've added link to this post and to rockedge's VBox-related post above to the HowTo list at User Contributions post:

http://www.murga-linux.com/puppy/viewto ... 29#1029029

wiak

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#633 Post by rockedge »

I am carefully looking over what you did and I will go ahead and repeat it plus modify what I have so far. I figured if I could boot it on VirtualBox it will boot on my physical machines and I don't need to restart my real machine every time I break something.

I should soon have a copy of something that works well and go from there

Plus I compiled pure-ftpd which is working really well for super easy FTP server set ups in WeeDog

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#634 Post by rockedge »

Success!

It was a faulty isolinux.cfg parameters. I used sr0 and it worked.
my structure is different slightly then what wiak used.

Bionic32-v8 as host mounted on /mnt/sdb1
mkdir /mnt/sdb1/weedog

copied the weedog build scripts with the firstrib00-32.plug into /mnt/sdb1/weedog and ran both scripts which successfully completed.
Deleted the scripts from /mnt/sdb1/weedog.
and copied from Bionic32-v8 and added to the weedog files:

Code: Select all

01firstrib_rootfs.sfs
efi.img
isolinux.bin
ldlinux.c32
libcom32.c32
libutil.c32
vmlinuz-5.2.17_1
initramfs05.gz
I added isolinux.cfg like this:

Code: Select all

DEFAULT linux
  SAY Now booting the kernel from SYSLINUX...
LABEL linux
  KERNEL vmlinuz-5.2.17_1
  APPEND initrd=initramfs05.gz bootfrom=/mnt/sr0 usbwait=12 inram_sz=100% changes=RAM	
then from a terminal opened in /mnt/sdb1 used:

Code: Select all

mkisofs -b isolinux.bin -c boot.cat -D -l -R -v -V "WeeDog" -no-emul-boot -boot-load-size 4 -boot-info-table -o "weedog32.iso" weedog
ISO is created in /mnt/sdb1 as weedog32.iso.

I created a VirtualBox machine and booted and logged in and startx activated the X desktop

I will be trying out isohybrid as well now, and construct some simple installer perhaps

this ISO is 1156 megs...But nothing has been stripped or compressed that much and I am not 100% sure all those files need to be in the ISO for booting...that I will have to look at

_
_

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#635 Post by wiak »

That's great rockedge. It's good to have weedog iso booting now, especially useful for virtual machine boots, I should try that. I used to use VMware a lot at work, and virtualbox occasionally at home but many years ago. Mainly I did a lot with User Mode Linux though since it was so lightweight back then and perfect for my network LAN/router experiments.

wiak

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#636 Post by wiak »

I've been thinking further about the point or otherwise of providing WeeDog (Void or otherwise) in the form of downloadable isos. To be frank, for the most part I see litlle point unless simply to get some people started.

Aside from removing var/cache/xbps contents I generally see no point in trying to produce a small download in terms of an iso. That has always been a major focus on this forum, but when I for example use BionicDog, though the iso download is small my daily system built using it isn't once I've installed the apps I want is several GB... If someone did want a small WeeDog for running on an old system wholly in RAM, then I'd suggest using BionicPup (or other Pup) kernel/firmware/modules sfs with WeeDog (you could certainly build a very small WeeDog that way); that has some disadvantages in terms of Void compatibility, but for old system, a good flexible approach since you still get xbps capability no traditional Pup would give you.

Alternative is to use a DebianDog, which is a great system in terms of functionality, though they are not particularly small in terms of firmware/modules either (I never use copy2ram with my DebianDogs) - Puppy on the other hand includes specially compiled kernel which to some extent needs less external firmware/module support. Of course 'special' kernel/module/firmware combinations could be built on and for Void itself - though personally I have no such plans.

The main bulk of Void Linux is simply the large amount of firmware (and to some extend the large number of modules) that come with it. In general use that just occupies disk space (of which there is usually plenty) - Void Linux as a distro is however extremely efficient in terms of RAM and CPU use, and hard to beat IMO. The one exception where smaller rootfs size would be useful would be when using copy2ram in a very small system configuration (more apps soon take up much more space than the firmware...) - certainly less firmware would provide a one-off overhead reduction, so if anyone comes up with a way of determining what firmware and modules their own system actually needs of course that would be a useful utility.

Otherwise, I think simply using the two build scripts is the best way to install WeeDog (or use an existing root-filesystem, Void, or Slackware, or whatever, along with WeeDog initramfs provided via build_weedog_initramfs script).

For special purposes, such as rockedge's Zoneminder, I think an iso of 600Mb to 1GByte (or more) is perfectly appropriate, or for an image to use with VirtualBox or similar (though it remains the case IMO that installation via build scripts is much more flexible in practice, in the likes of VirtualBox too - i.e. builds specially created firstrib build plugs... and then, if wished, put into bootable iso form via simple utility).

But above is just my opinion - if someone wants to pursue the making of a small iso that's fine of course!

I would always encourage using the build scripts (with well-developed firstrib plugin) for simplicity and ease of maintenance over downloading any pre-configured iso. It is just so simple and more satisfying overall in the end... The simplicity of the build system and the special functionality of WeeDog initramfs is a major point of difference to other offerings. That, and the ease also of adding additional WeeDog initramfs functionality via its simple script plugin support, which removes the need for expanding, custom one-purpose editing, and re-compression of the initramfs cpio archive itself. So anyone can practice adding functionality to WeeDog without danger of damaging the core initramfs itself - like a lego distro building kit, which gives it a huge amount of scope for developers, and, by its thus very open nature, also invites the contributions of others.

wiak

EDIT: But in terms of not making a small iso, I'm talking above about general purpose download iso. For your own use on particular system, see my next post, where I'll describe what I do sometimes.

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#637 Post by wiak »

Following on from my immediately above post:

Generally, I find I don't need much firmware. Immediately after booting WeeDog, I run command:

Code: Select all

dmesg | grep firmware
which only reports:

Code: Select all

[    8.997192] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[    9.049562] iwlwifi 0000:02:00.0: loaded firmware version 8.83.5.1 build 33692 op_mode iwldvm
I then google iwlwifi "firmware version 8.83.5.1 build 33692"

and find this link:

https://wireless.wiki.kernel.org/en/use ... rs/iwlwifi

and that page tells me that firmware 8.83.5.1 corresponds to:

iwlwifi-5000

as I once described here: http://www.murga-linux.com/puppy/viewto ... 22#1022822

Actually, for my wifi driver, I use firmware: iwlwifi-5000-5.ucode

So what I do, is I build firstrib_rootfs including the install of linux-firmware-network (which happens to include the above I think). Anyway, I then copy the whole of /usr/lib/firmware out of firstrib_rootfs (for use when needed) and simply put back firstrib_rootfs/usr/lib/firmware/iwlwifi-5000-5.ucode. That simple step saves almost 250 MB! i.e. start with almost no firmware and only add what you later find you need...

[at that stage you can also further prune the system. For example, emptying out contents of /var/cache/xbps and /usr/share: man pages, docs, info pages, and locales etc. and even stripping lib binaries, though sometimes risky and I wouldn't bother]

Great thing is, in WeeDog, I can boot with that uncompressed firstrib_rootfs, to make sure that firmware is enough... simply by renaming it to 01firstrib_rootfs (and disabling any 01firstrib_rootfs.sfs by maybe renaming that to NO01firstrib_rootfs.sfs). Result is, that for commandline use anyway, weedog boots fine with that and connects to Internet via wifi... Yes, I may need some other firmware for graphics use (maybe /usr/lib/firmware/i915 folder for my system (but I can add that later). Of course I could also make a small NNfirmware.sfs file for adding to one of the WeeDog layers if I so decide (but not really necessary), or an alternative firstrib_firmware_modules.sfs (as described in earlier builds).

Modules don't take up so much space as Void-provided-firmware anyway, but there is a simple script available that could be modified to prune down the modules you might need (based on output of lsmod after WeeDog booted). remove_kernel_modules.sh:

https://wiki.ubuntu.com/ReducingDiskFoo ... modules.sh

I haven't tried the above remove modules tool yet, but it looks sensible and simple enough.

So with these simple measures you could make a very small custom iso for your own system (which is handy for copy2ram use).

The following is a useful hw determining tool:

Code: Select all

xbps-install lshw
Some details in about such tools in this thread:

http://www.murga-linux.com/puppy/viewto ... 97#1022797

and: https://ezix.org/project/wiki/HardwareLiSter

wiak

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#638 Post by rufwoof »

In addition to clearing out /var/cache/xbps my firstrib plug also clears out

/usr/lib/libreoffice/share/extensions ... dictionaries for other languages

/usr/share/locale .. locales for other non used locations

rm -rf /usr/lib/libreoffice/help # 195MB (68MB lz4 -Xhc compressed)

A massive saving can also be achieved by clearing out firmware and modules. To achieve that you can compile the kernel using localyesconfig such that all of the modules required for your hardware are built into the kernel. Last time I ran that it expanded the kernel from something like 7MB to 16MB (minimal increase) whilst knocking down the main sfs filesize by around 250MB (big saving). BUT compiling the kernel whilst relatively simple does take a lot of time ... several hours (and some). And of course the product is a kernel that is specific to the hardware/machine (non portable).

Haven't tried it, but this looks helpful https://github.com/graysky2/modprobed-db. Rather than having to attach everything you might use prior to building the kernel with make localyesconfig you can record them as you go using that utility (assuming it works as described)
You need to make sure that all modules you will ever need are loaded at the point you run make localmodconfig. One tool that can help to achieve this is https://github.com/graysky2/modprobed-db.

First, boot up a default distribution kernel and run /usr/bin/modprobed-db store periodically, or every time you connect some new piece of hardware.

Then, run sudo /usr/bin/modprobed-db recall which will load all the modules that were ever loaded when modprobe-db store was run, and now you do make localmodconfig.
From the kernel documentation
"make localmodconfig" Create a config based on current config and loaded modules (lsmod). Disables any module option that is not needed for the loadedmodules.

To create a localmodconfig for another machine, store the lsmod of that machine into a file and pass it in as a LSMOD parameter.
For me however, saving 340MB of firmware/modules out of the final main sfs (lib firmware and modules folders), 250MB or so smaller sfs filesize is pretty much irrelevant on my 4GB laptop. Inconvenience of having to compile each kernel rather than having it delivered as a pre-built binary ... for zero benefit in my case. And from what I've seen so far, voidlinux regularly release kernel versions/changes.

My laptop sees around 700GB of ram reserved for graphics, so 3.3GB out of 4GB total ram of remaining available ram. I boot everything into that, copy2ram, changes=RAM with a main sfs that is loaded with pretty much all I use (chromium, libreoffice, vlc, audacity, openshot ....etc.) and typically that sees around half of the available ram remaining for operation. Just in case ... I also create/load a encrypted swap file on hdd (located on sda1), automatically sized to the same size as ram (3.3GB) and commonly even under pretty heavy usage not even half of that gets used (I should really revise the code to set the swap file to being half size). And that's all when using lz4 (-Xhc) compression, with xz high compression the size can be reduced substantially more - but at the expense of slower performance. voidlinux is quick by itself, when used with fast lz4 decompression and having everything run in ram (my laptop's spin up speed is noticeably slow, the reason I came by it (gifted (as good as new) by a Windows fan, but where Windows ran painfully slow on the laptop)) .. it really flies.

PS ... I boot with inram_sz=100% and thus far haven't noticed any downside from doing so.
Attachments
s.png
(8.71 KiB) Downloaded 422 times
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#639 Post by rufwoof »

I've removed all tray icons from jwmtray such as xload, cpu temperature, caps lock, battery power ... to just leave the clock. That considerably lowers the temperature and cycles, so improves battery reserve times. No menu button either (I set the jwm clock to left mouse click activate show/hide desktop, right mouse click to show the jwm menu).

Code: Select all

        <TaskList maxwidth="2560"/>

        <Dock/>
        <Clock format=" %a %d %b %H %M"><Button mask="1">showdesktop</Button>
        <Button mask="23"></Button></Clock>

    </Tray>
I leave jwm menu sparse however, just shutdown/reboot and launch Terminal, but where I have tilda terminal always running, F1 to show (full screen) and hide it. To launch programs I use xlunch, so just one file to maintain (/etc/xlunch/entries.dsv). I also have skippy installed, so you can quickly see/select open windows - but jwm tray also provides that function. I have alt-space and altgr-space (right side of spacebar) set to present xlunch, which once you're used to it is very quick/easy. /etc/system.jwmrc (being a single user desktop (laptop) I edit the main files rather than storing versions under /root) code snippet ...

Code: Select all

    <Key mask="C" key="Down">exec:amixer -c 1 set Master 2%- </Key>
    <Key mask="C" key="Up">exec:amixer set -c 1 Master 2%+ </Key>
    <Key mask="C" key="0">exec:amixer -c 1 sset Master,0 toggle </Key>
    <Key mask="4" key="space">exec:skippy-xd</Key>
    <Key mask="A" key="space">exec:/usr/local/bin/xlunch-show.sh</Key>
    <Key mask="5" keycode="65">exec:/usr/local/bin/xlunch-show.sh</Key> # AltGr space 
ctrl up/down arrows raises/lowers the volume.

The rightmost vertical strip of keys all serve as browser control, HOME jumps to the top of a web page, END jumps to the bottom, Page Up/Down jumps up/down by a page, arrow keys moves a web page line by line.

I also prefer mc (midnight commander) for file management over rox, but I do use rox (and geany) at times. mc is available through F1 (tilda), and where typically that is the first thing I load up in one of tilda's tabs (multiple terminals).

alt-space and type the first few letters of a program to filter down to the intended program, or use the arrow keys to move to the target program and press Enter to launch it ... or use the touchpad and mouse to and click the program to launch it. Very quick once you're driving that instinctively. And voidlinux's supplied version of xlunch works incredibly well IME. The best so far that I've come across (having tried xlunch across a range of different distros).

Image Image

Alt space and run the zzz command to put the laptop instantly into sleep mode whether the lid is opened or closed. Enter to wake it up again. I don't run any acpi stuff so if I have something running and I'm physically moving around I can close the laptop lid without it automatically going into sleep mode.

Highly recommend you try xlunch for yourself. I never used to use the AltGr key, as on a UK keyboard the only thing that I might occasionally use it for is to altgr 4 ... to print the Euro currency symbol €, other than that it was pretty much a dead key for me. Now as a right hand xlunch launch ... it gets more than its fair amount of usage :)
[size=75]( ͡° ͜ʖ ͡°) :wq[/size]
[url=http://murga-linux.com/puppy/viewtopic.php?p=1028256#1028256][size=75]Fatdog multi-session usb[/url][/size]
[size=75][url=https://hashbang.sh]echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh[/url][/size]

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#640 Post by wiak »

Well I think I have finally found a bug (introduced in model 05 I think), though I have to double-check. I haven't found it till now, because it actually has no effect on anything weedog initramfs is being used for currently. But it would have been noticed if using, say, BionicPup kernel and a 00firstrib_firmware_modules.sfs, which no one contributing to this thread has done for a long time. It's simply this line 325 of build_weedog_initramfs05_s203.sh script:

Code: Select all

	mount --bind 00firmware_modules/usr/lib/modules /usr/lib/modules  # needed for overlayfs module
Though, I have as yet to double-check this, I believe it should be:

Code: Select all

	mount --bind ${layers_base}/00firmware_modules/usr/lib/modules /usr/lib/modules  # needed for overlayfs module
I came across it, because, following my previous post, I was experimenting with extracting /usr/lib/firmware and /usr/lib/modules out of firstrib_rootfs and putting that into a 00firstrib_firmware_modules.sfs, which actually works by the way, even with the above bug because in Void kernel case the modules are in any case already loaded into the initramfs itself so above failing mount has no detrimental effect (would be different with BionicPup kernel case where modules are not pre-loaded into the initramfs...).

The reason I was experimenting was simply that, though not quite as good as re-compiling the kernel with modules/firmware built in, does allow using very small amount of system-specific firmware in the firstrib_rootfs. On my system, I just have that iwlwifi firmware in /usr/lib/firmware, so also getting a massive firstrib_rootfs size saving... (firmware is not preloaded into the initramfs, only the modules and these take up much less space than the huge amount of firmware Void provides).

I'll mention a bit more of how to use a 00firstrib_firmware_modules.sfs like this in my next post, once I've double-checked everything. It makes it possible to provide a much smaller downloadable iso, but with separate 00firstrib_firmware_modules.sfs, which may be no big deal at all. Of course if kernel changes, the initramfs needs rebuilt anyway since the modules needed in that also need changed. With Void kernel you certainly do need sufficient modules inside the initramfs itself such that it can do the overlay and also load the modules needed to see the external firstrib_rootfs on whatever media it is stored on (however, there is no doubt that modules list inside initramfs could also be pruned significantly - though I'm also not sure it is worth doing so for most scenarios on most computers 10 years old or less...).

wiak

Post Reply