Puppy Topdog

For talk and support relating specifically to Puppy derivatives
Post Reply
Message
Author
User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

Puppy Topdog

#1 Post by rufwoof »

Since 'TopDog' variant of puppy doesn't seem to have been used in quite a while (8 years) I'm slipping in within another variant of that same name. More of just a proof of concept at present - that seems to work.

I have a small PXE server installed on my home puppy PC. Most network cards have the option to net boot and PXE server is a means to deliver such boot up capability to other PC's on the same LAN. Some have pushed that further and gPXE from the 'cloud'. Typically to net boot you press the F12 key whilst the PC is first starting up and if set up in the BIOS that will present a bootup screen.

For PXE booting you need just the vmlinuz and initrd.gz file. Similar to when you boot puppy frugally using the likes of grub4dos. So as to pass puppy sfs along with those files you typically drop puppy sfs into the initrd.gz file. To do so I use the commands :

To extract the contents of initrd.gz
mkdir NewDir
cd NewDir
zcat ../initrd.gz | cpio -id

Copy the puppy sfs into that same directory
cp ../puppy*.sfs .

Reform the initrd.gz (note that you probably should make sure you have a copy of the original initrd.gz beforehand)

find | cpio -o -H newc | gzip >../initrd.gz

That new (larger) initrd.gz with the puppy sfs inside it can then be booted using grub4dos or PXE net booted etc. i.e. those same two files (vmlinuz and (larger) initrd.gz can also be used to boot using grub4dos so this isn't a PXE only exclusive concept.

A problem with PXE booting is that it tends to be slow. Better if we can use HTTP instead, at least for part of the bootup process. So I came up with the concept of layering one puppy (topdog) on top of another (underdog). For the underdog I opted to use Wary 5.3 XVesa Baby Barebone http://murga-linux.com/puppy/viewtopic.php?t=83841 which is a barebone XVesa only pup wth very limited functionality, but it does tend to boot straight to desktop and if you're hard wired to the net it also tends to auto connect. Being around a 60MB pup that PXE boots reasonably quickly. I've added code to that to then download a full-blown Puppy Wary 5.5 sfs from my googledrive ... and then sfs load that 5.5 puppy sfs. The only modification to that Wary 5.5 pup is that the Startup folder contents have been deleted as we don't want those to run when the puppy is sfs loaded (as we already have our tray, network ...etc already running in the underdog Wary 5.3 puppy).

CAUTION : I HAVE A FAST INTERNET CONNECTION AND NO BANDWIDTH LIMIT/CHARGE. This isn't for modem dial up (or other slower connections) or if you pay for bandwidth.

The code so far is without error checking and is very first-cut/basic. I simply added the creation of another folder in initrd (that I called linkexec), that contains a single script. That way changing the script is relatively simple i.e. extract the content of initrd.gz, modify the /initrd/linkexec/linkexec.sh script and then reform the initrd.gz file again.

To invoke that script when the underdog Wary 5.3 boots I simply added a call in its Startup folder. To ensure that its called only once per boot rather tha each time a Startup occurs (exit to prompt, xwin back in again) the script /initrd/linkexec/linkexec.sh script deletes the /root/Startup/linkexec.sh script as part of its processing.

The other main job of the script is to 'automatically' download the Wary 5.5 puppy sfs (134MB) from my googledrive (that's accessible to all) which it does using wget. Once downloaded it then sfs_load's that 5.5 puppy sfs (that has no Startup folder content). Thereafter the puppy is running 5.3 Wary XVesa only underdog, with 5.5 Wary full-blown on top and as such has access to Abiword, gNumeric and all of the other features contained within the full blown Wary 5.5. You can even exit to prompt, run xorgwizard and then switch from using XVesa to perhaps nvidia or whatever graphics server.

As I said the code is very weak. For instance there's no checking to see if the network is available before issuing the wget command ... etc. (Currently I've just coded a 30 second wait beforehand to give time for network connection to occur). When (if) the 5.5 Wary puppy sfs is downloaded and being sfs_loaded it prompts for where to run that from - I tend to answer with a use 'RAM' choice. Assuming your PC has a GB or two of ram space then that seems to work fine.

Another thing I've added to the Wary 5.3 XVesa barebone is dbus, together with a get_latest_firefox script in the /root directory. Run that once and it will download and run the latest firefox. To run a second time search down through something like /root/firefox-42.0/opt/firefox Even without 5.5 topdog loaded on top that firefox works ok, views youtubes etc (HTML5). I've even tried out (sfs loaded) local copies of Libre, Skype and master-pdf-editor etc and the underdog worked reasonably well.

Thoughts that come to mind is that with further development (encasing the no error checks etc in current scripts), having a consistent underdog pup that could then be modified to load different topdog pup's by just changing the pointer (where to download the topdog puppy sfs from) would be rather neat. "Hey - try out my topdog located at http://abc.xyz type of thing". The other thought is the HTTP booting the larger part of a PXE booted system is so much faster than local PXE booting a large image. For instance at my network speeds firefox downloads in seconds, whilst the above puppy wary 5.5 full blown sfs takes around 30 seconds to download.

A final word of caution. I've PXE booted several relatively older machines fine using Wary 5.3 XVesa Barebone however one relatively new (year or two) machine boots, but hasn't network nor mouse activated. I guess because of lacking more recent modules/drivers in that (now dated) Wary 5.3, so your mileage may vary. Another problem I've encountered is that XVesa is somewhat temperamental - I just stick with a bulk standard choice of resolution change initially from the 640x480 to 1024x768 and that works for me, but again your mileage may vary - and given the 30 second clock ticking before the current code starts downloading I'd suggest perhaps just booting and sitting back for a while without doing anything for a few minutes and perhaps wait for the sfs load to prompt to popup before moving.

vmlinux (2MB) https://drive.google.com/file/d/0B4MbXu ... sp=sharing

Initrd.gz (56.8MB) https://drive.google.com/file/d/0B4MbXu ... sp=sharing

Currently without appropriate code even though it may have downloaded Wary 5.5 sfs previously there's no detection of that. I haven't tried saving yet and run pfix=ram and opt to not save at the end of each session. Presumably with a savefile then it would already be set to load up fully from the get-go - but currently would start a download anyway despite having the 5.5 sfs already available.
Attachments
ss.jpg
(79.16 KiB) Downloaded 957 times
Last edited by rufwoof on Sat 07 Nov 2015, 11:46, edited 1 time in total.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#2 Post by rufwoof »

Condensed text version : Small puppy that automatically downloads another puppy sfs from the 'cloud' and sfs_loads that on top so that the functions of that overlaid puppy also become available.

gcmartin

#3 Post by gcmartin »

Hi @RufWoof. This is a great effort coming. Thx

Hope you account for the plethora of *drv files (namely adrv/fdrv/zdrv/bdrv/etc...drvr) that poliferate new PUP distros. A utilty to package the TFTP needs when using a distro's ISO could provide some ease of use that I think you are driving for. And, I assume/hope that there is some Puppy standard for the order that these should be stacked so that it eases your facility's use without individual investigations with questions when peripheral or applications dont behave after the PXE PC boots.

Hope this is helpful

P.S. One other distro name could be RemoteBootDog or ShepardDog to accentuate the service it provides to the LAN where this resides.

Or maybe this could become a package installable from via WOOFCE/WOOFQ during distros builds or PPM when distros run. It seems that DNSMASQ is showing up in few more distros, OOTB, to make your PXE facility installable with one less need for dependency to support your direction. This would allow the ability for your PXE boot PC facility to stay current within many/most new distros without an 8 year laps to rebuild the Topdog distro. And, I know you are aware how your facility would allow other subsystem integration with VM/KVM/Containers to boot a PC image directly from your facility no matter if is was running on a 32bit/64bit distro.

In any event, I look forward to testing to help

oui

#4 Post by oui »

Hi rufwoof

This seems to be near the concept of SliTaz. You can use (about) all old CD's made out Slitaz's ISOs (not only the actual ones!), and, at the invitation to enter boot values, enter the value

"web".

After that SliTaz offers you the possibility to start a lot of SliTaz versions, old and recent ones, directly from web to start them.

Very easy, but, on my internet, slow (SliTaz guy's did name short download times that I did never have myself, but, really, after long or short time, the wished version starts and probably will appear with it's login on your screen. SliTaz is now running in RAM... With it you can do one of both, use SliTaz as long as needing and shutdown (next session you will have to repeat each start the same procedere) or install one SliTaz version (can be a different version) on you resident memory / HD if the daily start from web is not fast enough for you.

If you use the stable version 4.0, you will use a system build following the principe of Russian dolls. The initrd.fs is divided into 4 separate units. The unit 4 is the hard minimal core console mode only, the unit 3 is the extension console mode, the unit 2 is the "justX" extension (graphical but nothing more), and the unit 1 is the extension to the normal graphical core including some really hightech services:

- own browser TazWeb based on webkit
- complete setting panel TazPanel based on TazWeb
- midori
- Xombrero can be installed (only 200 Kb download) as TazWeb, Midori as well as Xombrero are all based on webkit. The developpers of SliTaz did unify her packages to make it possible with only one unique Webkit environment :idea: and make all more external and more complicate for beginner (each expert did be a real beginner a day); it is one of the problems in Woof and WoofCE as well as, probably, woofCE vers. Trisquel...
- localhost in /var/web with total free access at disposition from user: you simply link somewhat in /var/web to make it easily accessible at your local web, localhost is a local web! Git's depositories are a real inconvenient for those working offline!

I did find this architecture best.

Perhaps a source of ideas for your new enterprise?

(My private interest would be to do the same from sources: start a Russian doll No. 4, add a kind of LP2-devx.sfs (like does Lazzy Puppy/RSH) to do more, recompile Russian doll No. 4, new version coming from sources, compile more, Russian doll No. 3 with the new compiled version and the actual old LP2-devx.sfs (alternatively rebuild immediately with the old a new devx before to continue), continue with Russian doll No. 2 and at the end of course Russian doll No. 1 (note: at this stage the actual SliTaz64 RC graphic core is only 44 MB "fat" :idea: ). Such a system would be completely coherent. And no Git or Mercurial: for this stage, the compilation would happen with sources from a sources tarball (last tarball from week or last night tarball) being entirely present on the user system through the localhost in /var/web (can be a link to a directory on harddisk or in an external memory as thick USB stick). Note: Tarballs prevent crashes resulting in changes in rolling distributions: you become never a really finished system. Ok, the inconvenient is, you have to live with the bugs included in the tarball. But a good history documentation would help...

actually, SliTaz doesn't use some layered file system. No LP2-devx.sfs is available nor possible, Present are SliTaz sources (this means sources stripped to build minimal system). Is good at the one side, bad at the other side because this needs an non universal source depot (better would be to to like Puppy today with binaries: take the stuff at the depository being more polyvalent like Debian/Ubuntu/Mint/Trisquel etc... or Arch/Archbang/Parabola etc... or Slackware... or Lfs/Blfs but this would be an automatic limitation: Lfs can not offer easily some universal stuff. If I remember my lovely software like the cartography app Merkaartor, Marble-Qt, wishing access to GPS etc., or Rosegarden, and a lot of more, it would be difficult to handle with Lfs/Blfs...

actually only an hand full of Puppy developers handles really with sources (manually or using T2)

but Puppy can handle with extreme raffines with external complete binaries environments like the perfect modular system from RSH that he did name SFS-Plus, like such a really needing LP2-devx.sfs (NuTyx does it without this stage; consequence: NuTyx have to compile tonnes of sources before getting a base permitting to do more...)

it would be so good to mix / unify those to system visions and add over them the abilities what you can find at T2, or Debian in apt-build + gdebi, or NuTyx build using an hand full of scripts from LFS sources...

.
Last edited by oui on Sat 07 Nov 2015, 00:03, edited 7 times in total.

Pelo

Bookmarked

#5 Post by Pelo »

bookmarked

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#6 Post by rufwoof »

I've uploaded a new version of initrd.gz that disables the get flash first-run (/usr/sbin/delayedrun) prompt as we're generally heading to see less of flash, more of HTML5 (that the likes of the latest firefox support). The new version also delays the screen resolution change screen until after the download of the Wary 5.5 sfs (so you're less tempted to exit/restart until after the process completes). I've also wrapped some code around the wget to test for net connection, sleep 2, test again ... until a network connection is detected (rather than the prior wait 30 seconds before wget more crude code).

Setting up a direct link to a googledoc file for wget to grab is a bit awkward, rather than deleting the existing file, uploading a new file and having to detect the new fileid and change the wget script to grab that different fileid, I found this handy tip about how you can manage versions of the same file (upload a later version, delete the prior version(s)), which keeps the same fileid http://www.opamingrazali.blogspot.co.uk ... drive.html

Attached image is from another PC on our LAN that I PXE booted and had up and running in a couple of minutes (including the time taken to download the Wary 5.5 sfs topdog).
Attachments
ss.jpg
XVesa set to 1280x1024x16 resolution
(96.93 KiB) Downloaded 804 times

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#7 Post by rufwoof »

Another version of initrd.gz uploaded that reduces the filesize down from 58MB to 56.8MB as gparted.bin, a 4MB file, has been removed from the underdog (barebone Wary 5.3 made a little bit more barebone).

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#8 Post by rufwoof »

Found out why the most recent PC on our LAN PXE boots to desktop but has no mouse/network - it has a Atheros pci/bluetooth hardware that apparently requires a alx module/driver that Wary 5.3/5.5 don't have.

When it comes to firmware/modules/drivers I'm at a loss.

User avatar
Puppus Dogfellow
Posts: 1667
Joined: Tue 08 Jan 2013, 01:39
Location: nyc

#9 Post by Puppus Dogfellow »

rufwoof wrote:Found out why the most recent PC on our LAN PXE boots to desktop but has no mouse/network - it has a Atheros pci/bluetooth hardware that apparently requires a alx module/driver that Wary 5.3/5.5 don't have.

When it comes to firmware/modules/drivers I'm at a loss.
rufwoof, have you seen this post:
01micko wrote:Slacko64 6.1.0.1

Updated 7 November

Changes are the same as the 32 bit version.

Get it: http://distro.ibiblio.org/puppylinux/te ... ko64/6101/ - (kernel source is in the slacko64 repo with the pets)

It is rather large because it has lots of firmware, however it is not all loaded by default. On the iso you'll notice a new file called fdrv_slacko_6.0.8.1.sfs. This is the extra firmware which is cross checked from the kernel modules so that no firmware that is not needed gets included.

The idea is that if all your hardware works then just don't load the fdrv or if you do and install to usb, hdd, whatever discard it. But if some piece of hardware, be it your graphics card or wireless card, doesn't work then you can load the fdrv from the kernel commandline :

Code: Select all

puppy pfix=fw_drv
or tack fw_drv onto your grub or syslinux commandline.

Other than that this is just a normal slacko64 in every other respect. So report things as per normal.
from here.

that help you out any? seems like you could offer tons of firmware and still keep the thing skeletal.

User avatar
rufwoof
Posts: 3690
Joined: Mon 24 Feb 2014, 17:47

#10 Post by rufwoof »

Thanks Puppus. I've just dropped Wary 5.3 Barebone XVesa into a 3.14 (Tahr) kernel .. PXE booting that on the latest PC has the network and mouse working as they should. i.e. resolved. Thanks anyway.

Whilst XVesa's not great functionally its great for consistency and seems to boot straight to desktop on everything I've thrown at it. Combined with a recent kernel that has broad support of hardware and the two are a good partnership for general initial booting and getting a network connection going - which in combination provides the means to download/install more appropriate choices for each individual machine.

Post Reply