Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 14 Nov 2018, 15:50
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
CLOSED : Underdog/Hybrid Debian Jessie (boot frugal or full)
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 3 [33 Posts]   Goto page: 1, 2, 3 Next
Author Message
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Fri 30 Sep 2016, 15:39    Post subject:  CLOSED : Underdog/Hybrid Debian Jessie (boot frugal or full)  

THREAD CLOSED

Barry outlined UnderDog in http://barryk.org/puppylinux/development/howpuppyworks.html ... extracts of which follow :

Quote:
An exciting alternative to the squashfs extensions is to use an existing installed Linux distro as the bottom layer:


Quote:
What the above diagram is intended to convey is that the bottom layer is a partition, not the "underdog.lnx" file itself. File underdog.lnx is just a text file, containing the name of a partition, for example "hda1".
At bootup Puppy will read underdog.lnx and will mount the partition as the bottom layer. If that partition happens to have a Linux distro installed in it, then the entire distro filesystem will "show through" on the top layer of Puppy's unionfs.


But noted in http://barryk.org/blog/?viewDetailed=02668
Quote:
There are possible problems, mostly to do with mismatched libraries. The Puppy libraries would be in a higher layer, which might mean that an application that you want to run in the Underdog Linux layer might not work. With some care though, it should be ok, for example with Wary/Racy, use an older version of Ubuntu not the latest as the Underdog. Or, better, use a matching distro, example Slacko and Slackware 13.37.


The image file available here is a form of Debian Jessie LXDE/Openbox (64 bit) UnderDog (of sorts, there's no underdog.lnx file involved as initrd and vmlinuz are as provided by Debian).

Its a xz compressed image file (1.3GB that expands to a 16GB USB image) so you'll need a USB of a least 16GB and a means to install the image to that USB http://barryk.org/news/?viewDetailed=00330 and have your PC set up so that it can boot that USB http://barryk.org/quirky/uefi.htm. For Linux users, you can use dd to write the image to USB :

Be careful, write to entire drive, for example 'sdf', not to a partition, example 'sdf1'.
Make sure that the drive is unmounted, as some Linux distros will auto-mount it as soon as it is plugged in.
Also be extra careful that it is the desired USB drive you are writing to, not your main hard drive!
Code:
xz --decompress --stdout jessie-usb.img.xz | dd of=/dev/sdf bs=4M conv=fsync

Be patient, it might take a hour or more to write to USB.

BOOT CHOICES

1.

Boot loader → vmlinuz → initrd → Debian Jessie bottom layer + main sfs (empty by default) + additional sfs's/.dir/.ext layers if added.
Changes stored in ram, which can be flushed to the bottom (persistent) layer using flush2disk (2nd icon on panel), or are lost at shutdown/reboot.
Layering achieved by using persistence persistence-read-only boot parameters

2.

Boot loader → Debian Boot loader → vmlinuz → initrd → Debian Jessie
Changes stored as and when they occur. Comparable to a Debian full install.


Same vmlinuz and initrd in both cases, as supplied by Debian

Benefits of type 1 is that it largely circumvents that USB's are slow to write to and their life can be reduced by excessive/repeated writes. Another benefit is that changes can be 'undone' simply by rebooting without saving (handy for trying out things). Disadvantage is that as changes are stored in memory (with option to flush those changes to disk upon demand) which tends to be more limited compared to free disk space.

Type 2 in writing changes to disk as and when they occur is constrained to the number of changes only by available free disk space. Disadvantage is that it runs slower than type 1 that records changes in memory and that all changes are preserved no matter what.


By default auto logs into desktop as user

Passwords :

user = live
root = me

Note that pcmanfm (file manager) has a right click option to open a window as root (right click on folder, select "SU file manager" and that window opens under root userid).

If you only install programs from Debian, using apt-get (text) or Synaptic (gui), then you partake in a large central system of stable programs that work well with each other. Security issues/updates from Debian tend to be quickly addressed/applied/distributed, and provided you update periodically then over time you'll automatically be migrated onto later releases/versions.

The img does contain non-free software/firmware, so please inspect and confirm whether or not it is ok for you to use such software in your country/region. Downloading and usage is on the basis that you acknowledge, respect and honour each individual components Copyright and Conditions of Use.

With thanks and greatest appreciation going out to Barry without whom such projects would never have seen light. And of course to Linus Torvalds. And also to the extensive Debian team who tirelessly develop and maintain Debian. Thanks also to all those who helped to develop, maintain and enhance snapmergepuppy/save2flash (and to Fred who helped me fork that for this particular project). Thanks also go out to all those others such as the LXDE and Openbox teams ... amongst many others, who have helped make Linux what it is today.
sc.gif
 Description   
 Filesize   55.92 KB
 Viewed   745 Time(s)

sc.gif


Last edited by rufwoof on Sat 15 Oct 2016, 16:40; edited 11 times in total
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Fri 30 Sep 2016, 15:42    Post subject:  

Technical background - How Built :

Debian Live CD /live and /boot folders copied out of the ISO onto HDD and booted frugally using grub4dos. Updates/additions (programs)/cosmetically changed and saved where the save 'folder' is the same partition as where grub4dos and /live /boot were installed (note uses a save partition, not a save file or save folder). Largely turning a livecd version (as on the livecd) into a more full install version (/etc/apt/sources.list extended from the limited livecd repository set to a full set). Set to boot using SystemD using persistence persistence-read-only boot parameters for the frugal boot choice, or as a more conventional full install for the second boot choice (that chains to Debian boot menu that offers three boot choices).

All of the filesystem.squashfs (main squashed file system) was then extracted and merged into the save 'file' (partition) ... in effect moving everything into the save file, leaving no need for the main filesystem.squashfs. A near empty filesystem.squashfs has been left however as that better enables loading of sfs's if so desired. sfs's can be loaded by dropping them into the /live folder and listing them in /live/filesystem.module (they are loaded/layered in the order specified in that file). Alternatively if filesystem.module is deleted (or renamed) then all files with a .squashfs suffix are loaded in alphanumeric order.

To facilitate saving changes recorded in memory during a frugal boot, flush2disk script (amended version of save2flash) was added.

The HDD installed version (partition) was then replicated on a USB, and that usb was then used to form the img.xz (after zero'ing free space to aid better compression).

The initrd.img file as supplied by debian is a .gz compressed file. To extract the content make a copy of that file with a .gz suffix and then uncompress that file. Make a folder beneath where that file is, cd to that folder and run cpio -id <../initrd (or whatever filename you decompressed to). To reform use find | cpio -o -H newc >../initrdnew and then cd .. and compress that file using gzip (gzip initrdnew) and finally add a img suffix (mv initrdnew initrdnew.img).
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Fri 30 Sep 2016, 17:13    Post subject:  

Operationally the default (frugal style) boot can be as quick as a HDD installed version. Writes to disk (usb) however are the speed killer, so any saves or booting full mode (that writes all changes to usb as they occur) are slow. Faster USB 3 would help in that respect. As such applying updates (apt-get update; apt-get upgrade), and even saves, are best applied whilst you have other tasks to do.

So best run in frugal mode, saving docs/files on another partition/disk. Or perhaps creating a direct link to the lowest layer for document storage :

su (and enter 'me' password)
cd /home/user
mkdir /lib/live/mount/persistence/sdf1/Documents
ln -s /lib/live/mount/persistence/sdf1/Documents Documents-persistent

type command (changing sdf1 accordingly), which for 'user' will create a Documents-persistent folder (symbolic link) in the home folder that writes directly to the usb (preserved across reboots, even if no save is made).
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 07:02    Post subject: HDD installation  

If you decide to install to HDD you can't just copy the files from usb to hdd (there are symbolic links and permissions that just wont copy correctly and you'll just have a broken system). Instead use a means that preserves those links/permissions.

Personally I tend to create a sfs of the files/folders and then extract that sfs, but as you can't extract a sfs to the root folder I use a temporary folder and then use a file manager to drag/move all of the contents of that folder up to the root level.

If for instance the usb is mounted as /mnt/sdf1 then change directory to the root of a newly created ext3 or ext4 HDD partition and

mksquashfs /mnt/sdf1 jessie.sfs

once that completes and you have /jessie.sfs on the HDD then

unsquashfs jessie.sfs

which creates a /squashfs-root folder on the HDD with all of the files/folders. Then use filemanager to open up that squashfs-root folder, select all files (including the 'hidden' files) and drag those up to the / level and select the 'move' option. The empty squashfs-root folder can then be deleted.

That will still require grub4dos to be installed (unless you have it installed already), so run that, but don't create a menu.lst file (or make a backup of the existing menu.lst and replace the new one with that backup after running grub4dos installation). Or if you already have grub4dos installed then edit the existing /menu.lst file content to include the entries contained within the jessie-usb menu.lst file.
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 07:10    Post subject: Firewall (UFW)  

A note about ufw (firewall). When you click on the panel shield icon for ufw the values are all greyed, and you have to click unlock in order to see the actual current status of the firewall (by default its sets ON, even though the greyed out initial window looks as though its set OFF). That's just the way, for whatever reason, ufw works.
Back to top
View user's profile Send private message 
backi

Joined: 27 Feb 2011
Posts: 1563
Location: GERMANY

PostPosted: Sat 01 Oct 2016, 07:25    Post subject:  

Hi rufwoof aka captain Kirk !

Phoenix/Sirius rising ...
Maybe call it--- Sirius Puppy-The Greater Dog .

Or Debian-Dog-Star .

" Sirius – in the constellation Canis Major the Greater Dog – is the sky’s brightest star. It’s very easy to spot on winter and spring evenings. "

http://earthsky.org/brightest-stars/sirius-the-brightest-star
https://en.wikipedia.org/wiki/Sirius

There are also options for names Sirius A........or Sirius B .

"What the naked eye perceives as a single star is actually a binary star system, consisting of a white main-sequence star of spectral type A1V, termed Sirius A, and a faint white dwarf companion of spectral type DA2, called Sirius B.

This is advanced Puppielogie .

Greetings from Mother Earth to captain Kirk aka rufwoof on Star Ship Enterprise in Galaxy Linux .

May the Force be with you
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 08:13    Post subject:  

My fine fellow, follically challenge as I am would equate me more to Picard

VY Canis has a nice canine ring to it and perhaps the better choice Smile In size, dwarfs the earth bound pup's.

[That video is dated BTW, Canis is no longer the top dog].

/usr/share/locale is nigh on 400MB alone!
Loads of firmware ready to go
Bluedevil (bluetooth)
Libre office suite
Skype
Master PDF editor
Firefox-esr
Chromium with PepperFlash
Openshot (video editor) together with Blender for 3D animated titles, full inkscape
Audacity for sound editing
Fred's - Dog Radio (Dance Groove is my favourite background music channel)
Pulse Audio
Record my desktop
mtpaint/gimp ..... etc.

All updated to the latest Debian Stable version at the time of uploading.

There's also a full documentation set, for instance if in a terminal you type
man live-boot
then you'll see the manual entry for live-boot.

If you boot full (no sfs's loaded) and run
apt-get update
apt-get upgrade
you'll be updated with the latest security fixes (and program updates). Which might include kernel updates also.

(full image here)

Providing you stick with Debian repository of stable programs that all work well with each other/together, then installing/removing programs is relatively easy using Synaptic gui pack manager. And being able to boot read only means you can try programs out (install them), and if no good, a reboot undoes all of that trial.

Given Debian's Toy Story naming, Wheezy -> Jessie -> Stretch, I guess Kirk/Picard is out of the window (airlock).

Last edited by rufwoof on Sat 01 Oct 2016, 09:51; edited 3 times in total
Back to top
View user's profile Send private message 
backi

Joined: 27 Feb 2011
Posts: 1563
Location: GERMANY

PostPosted: Sat 01 Oct 2016, 08:38    Post subject:  

Seems i am a bit outdated (excuse me.... living on the countryside )
It looks like you`re quite well informed ....... .... me impressed again Very Happy
By the way.... do you refer to ......Jacques or his father Auguste Piccard ?...... Very Happy Very Happy Very Happy just kidding Smile Smile


O.K maybe call it Debian Super Dog .....or Pluto the Dog or...or..or.....
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 08:56    Post subject:  

Mouse into the bottom left corner and it might (depending upon your graphics card/setup), morph a desktop full of overlapping windows ... into a nicely spaced out set of windows where you can use left, middle (or depress scrollwheel), right mouse actions to focus a windows, minimise or close windows.
a.jpg
 Description   
 Filesize   58.76 KB
 Viewed   577 Time(s)

a.jpg

b.jpg
 Description   
 Filesize   29.6 KB
 Viewed   599 Time(s)

b.jpg

Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 09:21    Post subject: Executing a script at startup  

LXDE is a bit 'odd' for adding startup scripts

Open file manager (pcmanfm) and navigate to /usr/local, then right click on bin and select the SU file manager choice so that you open up /usr/local/bin in a su file manager window (pcmanfm puts a little icon near the top right to show its running as root, in contrast the likes of Thunar put up a total window wide orange banner, which personally I find a bit OTT).

Create a script in /usr/local/bin and make it executable. Then navigate to /home/user/.config/lxsession/LXDE and in autostart at the name of that /usr/local/bin script with a @ prefix, i.e. the default set looks like

@lxpanel --profile LXDE
@pcmanfm --desktop --profile LXDE
@xscreensaver -no-splash
@remount-rw
@brightside
@python-preload.py
@libreoffice --quickstart
@pre-save

At reboot those are run with root authority, so there's no need to have su checks in the /usr/local/bin script.
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 09:42    Post subject: SystemD service  

jessie-usb runs under systemD so you can run commands (as root) such as
systemd-analyze
that will show boot up timing or
systemd-analyze blame
that shows more detailed timings of individual services

To plug something into systemD, let's say run a test for a file in /tmp and if present do some action, you create a script in /usr/local/bin
Code:
   
#!/bin/bash
if [ -f /tmp/dosomethingatshutdown ]; then
  echo hello >/tmp/hello
fi

(Obviously with the code you actually want to run)

and then create a systemD service to link that into systemD i.e. create a .service file such as

/lib/systemd/system/dosomethingatshutdown.service
Code:
[Unit]
Description=do something at shutdown if /tmp/dosomethingatshutdown file exists
DefaultDependencies=no
Before=shutdown.target reboot.target halt.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/dosomethingatshutdown

[Install]
WantedBy=halt.target reboot.target shutdown.target

... and enable it

systemctl enable dosomethingatshutdown.service

you can check its status is enabled by scanning through

systemctl list-unit-files

that shows all files, or just look for the single file by name

systemctl list-unit-files dosomethingatshutdown.service
that typically reports back something like :
UNIT FILE STATE
dosomethingatshutdown.service enabled

Or you can disable it
systemctl disable dosomethingatshutdown.service

Of course all of the above should be using root (su).

If you can read quick enough, you might even see that service being checked during shutdown amongst all of the other textual messages logged to screen.

One thing I forgot in the uploaded version was to re-enable the ModemManager service. I have no modems so I disabled that bootup service, but if you do have a modem then you may need to re-enable it

systemctl enable ModemManager.service

The tricky part, at least for me, is the .service file and knowing how to code it to slot the execution into the right place, Before, Wanted By, After type values to enter. I guess to be more proficient in that you have to have a better understanding of all of the other .services that are running/available. I simply got the basics for the above example from browsing around/searching for examples.
Back to top
View user's profile Send private message 
backi

Joined: 27 Feb 2011
Posts: 1563
Location: GERMANY

PostPosted: Sat 01 Oct 2016, 10:06    Post subject:  

Me thinks..... you Captain Kirk did press the Hyper-Drive Button ......or are you even in Warp Mode ?

Oh... where are ? .....Ground Control to Major Tom ......

Seems " Major Elvis Kirk Piccard Tom " has left the building . ooohps... i meant this three Dimensional Universe .

" And If I don't meet you no more in this world ........I'll meet ya on the next one .... don't be late...
Don't be late "

Jimi Hendrix ....Voodoo Child
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 10:24    Post subject:  

Time warped ... and back again. Seen the future, where sfs's are less necessary, storage and transfer rates much quicker. Where even GB's are lost in available ram space Smile

Got-to-go, need to see a doctor ... whatshisname - forget, I remember the place though, near Gallifrey.

Quote:
" And If I don't meet you no more in this world ........I'll meet ya on the next one .... don't be late...
Don't be late "

Jimi Hendrix ....Voodoo Child
Back to top
View user's profile Send private message 
backi

Joined: 27 Feb 2011
Posts: 1563
Location: GERMANY

PostPosted: Sat 01 Oct 2016, 10:28    Post subject:  

Wwooowww ........ Rolling Eyes Rolling Eyes Rolling Eyes

Let me call it Cyber-Shamanism .
Back to top
View user's profile Send private message 
rufwoof

Joined: 24 Feb 2014
Posts: 2628

PostPosted: Sat 01 Oct 2016, 11:22    Post subject: .dir .ext loading at bootup  

There's a undocumented feature that Toni (Saintless) noted by inspection of /bin/boot/9990-misk-helpers.sh i.e. within initrd) where if you create a folder with a .dir suffix in /lib/live/mount/persistence/sdf1/live (replacing sdf1 with whatever your setup uses), then at bootup that folder will be 'mounted' under /

Similar to loading a sfs, except a conventional directory content layered in.

Any changes made to the / version of that, for example if I create /lib/live/mount/persistence/sdf1/live/test.dir/abc.txt and reboot then that will appear as /test.dir/abc.txt but changes wont be preserved unless you save the session (flush2disk, second icon in the panel). Or you could edit it directly as /lib/live/mount/persistence/sdf1/live/test.dir/abc.txt so that changes are preserved even without saving the session.

The other undocumented cases is that it will also load .ext2, .ext3 or .ext4 i.e. complete filesystems.

Unlike sfs's that have a maximum of 7 coded in by Debian, you can load as many .ext or .dir as you like. I haven't really experimented with that but looks like you might be able to create something similar to read/writable sfs type 'things' to load at bootup (at least in the case of .dir)

A observation however is that whilst if a filesystem.module file exists in the live folder then only squashfs files listed in that file are loaded at bootup, in the order specified in that file; If filesystem.module doesn't exist then all .squashfs files in the live folder are loaded at bootup in alphanumeric sorted order; Even if a filesystem.module exists however and doesn't mention a .dir folder, then that .dir folder still gets loaded at bootup.

Just as a reminder, you can create file filesystems using something like
dd if=/dev/zero of=somefile.ext4 bs=1M count=1024
mkfs.ext4 somefile.ext4

and then mount it using something like
mkdir mountpoint
mount -o loop somefile.ext4 mountpoint

and don't forget to umount it when done
umount mountpoint

In addition to .dir and .ext2/3/4 you can apparently also use .jffs2 (which I know nothing about). i.e. /bin/boot/9990-misk-helpers.sh 'looks for' *.squashfs|*.ext2|*.ext3|*.ext4|*.jffs2 when scanning for things to 'load'.

Being undocumented, a concern is however that Debian could upgrade the kernel/initrd in a manner that excludes that code.

Last edited by rufwoof on Sat 01 Oct 2016, 15:04; edited 1 time in total
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 3 [33 Posts]   Goto page: 1, 2, 3 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0897s ][ Queries: 12 (0.0140s) ][ GZIP on ]