Creating a truly headless pup derivative (new issues

For talk and support relating specifically to Puppy derivatives
Message
Author
ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

Creating a truly headless pup derivative (new issues

#1 Post by ldolse »

I'm working on a puppy derivative that can operate in a fully headless environment, from the initial boot onwards. At the same time I also want this derivative to be able to use X automatically if the system is actually hooked up to a monitor/keyboard.

A little background. These questions are focused on the tweaks I'm making for the next revision of mpdPup. As an audiophile oriented release I don't want unnecessary processes running, so I generally don't want X. This derivative is aimed at few audiences:
  1. Windows/Mac users coming from an iTunes/Mediamonkey/Foobar/etc world - CLI and config file phobic, so give them a GUI during the initial setup & whenever else required
  2. Technically oriented users who appreciate someone else doing the heavy lifting for them, but aren't afraid of a CLI (prefer remote management)
  3. (stretch) Users using embedded x86 boards such as the Alix or Soekris boards - most of which lack a VGA out
There are a few 'how to use puppy without a monitor' threads, but they're more about workarounds than getting this done as a standard part of a build. Most of the changes are around the initial bootup and shutdown.

Bootup
  • Detecting if a monitor is attached. Using ddcprobe right now. This is probably the most unwieldy piece, but it works on my hardware (except VMware which is quite vague about monitor support). (done)
  • Initial boot rc.country locale wizard - modified rc.sysinit to use the above script Not working with recent woof snapshots
  • Prompt the user to change the password if sshd is enabled - do it either from the ssh session or a monitor/keyboard session (done)
  • SSHd - start in rc.d/rc.services instead of init.d (done)
  • IP - dhcp default enabled in rc.network (done)
  • Bonjour/Zeroconf so a user can find the server (done)
Shutdown
  • Modify rc.shutdown to check if the tty that initiated the reboot is an SSH tty, and redirect output there instead of /dev/console.(done)
Initial implementation attached. I don't claim to be a Linux expert, so I don't know if what I've done is the 'right' way, and I'd love to hear suggestions for improvement. I've attached avahi as a separate package as many Windows users may not find it valuable. I did go out of my way to compile a very lightweight version of Avahi, by default they want to compile with a dozen dependencies....

If you install Avahi then your hostname is most likely puppypc.local, as that's the default Puppy Linux hostname for most builds.

Note I haven't tested these pets explicitly - I've tested all the files, but they were extracted from the setup I'm using to build my remasters, let me know if something seems to be missing.

All this work was done on 4.2 Turbopup. I think most of these scripts haven't really changed for the 5.x branch, but I haven't done a diff to confirm. The only thing pending testing is whether this will work from a serial console.
Attachments
avahi-daemon-0.6.30-i486.pet
Avahi/Zeroconf Daemon for autodiscovery using mdns
(175.29 KiB) Downloaded 514 times
Last edited by ldolse on Tue 22 Nov 2011, 00:37, edited 11 times in total.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#2 Post by ldolse »

Done a bit of experimenting. Using xrandr to detect a connected monitor doesn't work until after the first time X has been configured/run. So that's out. Looks like the other option is ddcprobe - problem is I can't find any decent documentation on this, and the Google examples I have found seem to show it's output can be all over the map.

Some examples:

Code: Select all

VMware Fusion (virtual monitor can't be disabled):
vbe: VESA 2.0 detected.
oem: V M ware, Inc. VBE support 2.0
memory: 64000kb
mode: 640x400x256
mode: 640x480x256
mode: 800x600x256
mode: 1024x768x256
mode: 1280x1024x256
mode: 320x200x64k
mode: 640x480x64k
mode: 800x600x64k
mode: 1024x768x64k
mode: 1280x1024x64k
mode: 1600x1200x256
mode: 1600x1200x32k
mode: 1600x1200x64k
noedid


Intel Atom board, no monitor:
vbe: VESA 3.0 detected.
oem: Intel(r) 82945GM Chipset Family Graphics Chip Accelerated VGA BIOS
vendor: Intel Corporation
product: Intel(r) 82945GM Chipset Family Graphics Controller Hardware Version 0.0
memory: 7872kb
mode: 1280x1024x256
mode: 1280x1024x64k
mode: 1280x1024x16m
mode: 1024x768x256
mode: 1024x768x64k
mode: 1024x768x16m
mode: 640x480x16m
mode: 800x600x64k
mode: 800x600x16m
mode: 640x480x256
mode: 800x600x256
mode: 640x480x64k
edid: 
edidfail


Intel Atom board with a Dell Monitor:
vbe: VESA 3.0 detected.
oem: Intel(r) 82945GM Chipset Family Graphics Chip Accelerated VGA BIOS
vendor: Intel Corporation
product: Intel(r) 82945GM Chipset Family Graphics Controller Hardware Version 0.0
memory: 7872kb
mode: 1280x1024x256
mode: 1280x1024x64k
mode: 1280x1024x16m
mode: 1024x768x256
mode: 1024x768x64k
mode: 1024x768x16m
mode: 640x480x16m
mode: 800x600x64k
mode: 800x600x16m
mode: 640x480x256
mode: 800x600x256
mode: 640x480x64k
edid: 
edid: 1 3
id: a028
eisa: DELa028
serial: 31455648
manufacture: 30 2007
input: sync on green, analog signal.
screensize: 38 30
gamma: 2.200000
dpms: RGB, active off, suspend, standby
timing: 720x400@70 Hz (VGA 640x400, IBM)
timing: 640x480@60 Hz (VGA)
timing: 640x480@75 Hz (VESA)
timing: 800x600@60 Hz (VESA)
timing: 800x600@72 Hz (VESA)
timing: 800x600@75 Hz (VESA)
timing: 1024x768@87 Hz Interlaced (8514A)
timing: 1024x768@75 Hz (VESA)
ctiming: 1152x864@75
ctiming: 1280x1024@60
dtiming: 1280x1024@70
monitorrange: 31-80, 56-75
monitorserial: TP21177O1EVH
monitorname: DELL E198FP
If I only want to target 'real' hardware then looking for ddcprobe output that contains 'monitor' or 'timing' seems to indicate a monitor is attached, but it's not so cut and dry for VMWare. I don't really need to support VMWare behaving perfectly but it makes me wonder if there is other legit hardware that will look similar to VMWare...



I also did some experimenting with SSH and rc.shutdown. Triggering a reboot from ssh seems to do some weird things. Most of rc.shutdown is NOT redirected to the ssh tty, but it seems like some parts probably are. When issued a reboot from ssh to my vm in Pupmode 5 I got the first dialog in the shutdown script, but selecting one of the options gave me back the prompt in the VMWare monitor while the SSH session was cleared and hung.

I'm not really sure how much work it would take to make rc.shutdown be able to redirect to either the ssh tty or /dev/console based on where it was initiated from. Beyond that there is the matter of figuring out which tty is tied to ssh...

User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#3 Post by rcrsn51 »

I don't know if it will help in your situation, but in another discussion about ssh and headless machines, the following shutdown code was used:

Code: Select all

nohup poweroff & 

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#4 Post by ldolse »

That could help for some modes of shutdown as I work through all the variables, thanks.

I just read through rc.shutdown and realized that almost everything was hard-coded to output to /dev/console.

I just did a quick tweak to add a check to see if the current tty is the same as the SSH_TTY environment variable, and if so set all the instances of /dev/console to that tty. Puppy wasn't running in Pupmode 5, which is what I really need to test, but the change in logic seemed to redirect everything to my ssh session, so I think this might be the route to take.

Edit - my changes to rc.shutdown seem to work perfectly over ssh with a fresh/pupmode5 install. Just need to solve the monitor detection, and I think that can be done relatively easily (just not sure how reliably).

gcmartin

#5 Post by gcmartin »

Hi @ldolse

This looks like an exciting contribution to this community..... and Linux n general.

That is, "the ability to get an Puppy desktop on a separate computer from the actual".

Thus, this indicates that one can have a Puppy system running somewhere supporting user(s) which logon for any number of reasons. Reasons: maintenance, upgrades, tool use, checking email, etc.

One question that I have which you may already be thinking about is how will you handle and receive audio/video on the client PC which connects to your headless (or even a "heady") PC?

Just curious.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#6 Post by ldolse »

Thanks gcmartin, note I'm not trying to create multi-user Puppy, just want to be able to setup and access the system without explicitly requiring a monitor/mouse/keyboard for the initial setup (or ever).

That's not to say you couldn't have multiple users access the Puppy desktop once this work is done - a remote xwindows session can run over ssh - whether this currently works with Puppy isn't something I've tested and not really on my to-do list. Although having a headless server isn't required for this application, you just need a working ssh daemon.

Based on that I'm not really thinking about remote users getting their own audio streams from the desktop. I believe something like that should be doable with Jack or Pulseaudio, but I'm not certain.

I do plan to make mpdpup easily create multiple sound outputs in the future, so you could potentially use it to create a whole-house audio system (provided the right audio cables were routed around the house).

SBP
Posts: 43
Joined: Wed 06 Apr 2011, 15:45

#7 Post by SBP »

Hi

This is really nice and will be very handy. I have the following suggestions.

1. Automatically detect and mount USB-Harddisks when inserted

2 Will you include samba, so that we can we can add new media files to the headless client?

3. A DNLA server.

4. For me a Squeezeboxserver would be optimal for streaming music.

gcmartin

#8 Post by gcmartin »

ldolse, Thanks for your reply.
I think I may not have conveyed my question approprately. I apologize.

When I said user (s), I was referring to people sitting at another PC on your LAN.

When I refer to desktop, a desktop to me means the Puppy desktop we see if we were sitting at that Headless PC with a monitor-keyboard-mouse-speakers attached. This has been a source of discussion in the forum at various times.

If I understand your answer: No, you will not be attempting to provide a multimedia desktop experience when you logon to your headless PC from another PC on your LAN.

Question:
Would you share how you provided the SSH setup, BOTH, for your Headless "server" and your thin/thick client so that we can replicate your contribution?

@SDP does ask key questions.... What ways do you envision your use of your Headless system at your location?

Thanks

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#9 Post by ldolse »

No worries, I think I did understand your question, just wanted to be sure. As far as I'm aware there are two basic ways of doing that - run vnc, or run a remote X session over ssh. I think the remote X session requires a fair bit of work, which is why things like VNC are popular.

For myself the primary use is as a music server - i.e. the replacement to the traditional CD player. One doesn't expect to need to hook up a monitor and keyboard to their CD player, same goes for a music server.

I do plan to upload a pet file with all my modifications once I get all the things targeted in the first post done. That way if someone else is targeting a different embedded application they can just run with it. I'm also hoping to get a bit of feedback and any advice for improvement.

@SBP, some of those things are on my to-do list.
  • Detecting and mounting USB is on the list, but a fairly low priority - I'd like to see some example implementations.
  • No plans to include Samba, I'd rather store the files on a NAS, and I want the system focused on playing music, not transferring files.
  • Squeezebox is still on my to-do list, I haven't forgotten, I just put it on the back burner. You'll be happy to know I got it working (at least starting without errors). However the build process is a nightmare, so it's going to take some serious effort to figure out how to package all the related files into a .pet or .sfs.
  • Not planning on DLNA, I have yet to be convinced of the value of the protocol. I'm open to being convinced about it though :)

SBP
Posts: 43
Joined: Wed 06 Apr 2011, 15:45

#10 Post by SBP »

Hi Idolse

I tried for some time back also to make a headless puppy - see these posts: http://murga-linux.com/puppy/viewtopic. ... 08&t=66633

So it can be done - however I did not have it to automatically show if you plugged in a monitor.

For the shutdown I used "Here is how I shut it down after a advice from 01micko. I installed acpid-1.0.8-20100603.pet and now I just press the power button once, then Puppy shut down nicely"

I that way you only need to press the powerbutton - as you would on any other equipment like a cd-player or TV and the unit would power-down.


Your concept is really good, but as I would use it would be like a CD-player, where you plugged-in a USB-HDD, and the played music from here. Therefore I think it would be cool if you also where able to put new music on the HDD via another PC or even a smartphone - therefore SAMBA would come in handy.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#11 Post by ldolse »

Booting without a monitor already works in mpdPup as xwin is disabled by default. I wanted the auto-detection of the monitor to eliminate the requirement for a less technical user to type 'xwin'. :) (And also to bypass the mouse/keyboard prompts which is probably more important.)

Regarding shutdown I don't think the ACPI pet is always required - the system is already running read only and clicking the power button shuts it down just fine. The main issue was getting the shutdown script working correctly on the initial bootup, and I'm now past that hurdle.

I understand where you're coming from with plugging in drives and automatically playing off of them. This is indeed something I'd like to get working, but I think it's easier said than done. Getting a drive automounted isn't that complicated. It's the unmount that's problematic - in my experience linux doesn't behave well when a mounted path is removed without being umounted, and how to you tell a system with no directly attached user interface you want to safely remove a drive? I know a technically oriented user can do it via SSH, but that's not particularly idiotproofed.

I know that people have solved this problem, as this is how the Bryston BDP-1 works, along with the Auraliti systems. I'm just not sure how they've done gone about it...


Samba can always be installed from a .pet - the config isn't locked in stone. I assume someone else has compiled it somewhere...
Last edited by ldolse on Tue 06 Sep 2011, 18:13, edited 1 time in total.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#12 Post by ldolse »

Just updated the first post with my initial draft of the changed files to support this. Feedback welcome.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#13 Post by ldolse »

Did some more testing with those files, turned out the script to detect whether a monitor is connected wasn't always working correctly with get-edid, so I switched it to using only ddcprobe. The side effect is that any 'monitors' that don't report their existence (e.g. VMWare) won't be detected. Not sure if VMware is the only culprit or if there are real world monitors which behave similarly.

I also read through rc.shutdown in more detail and decided to move the portion of the script which disabled the network and added a line to kill sshd. This wasn't actually causing me problems during my testing, but based on the comments in the script it appears that it might have caused problems for other users, as I think the ssh daemon was preventing the script from shutting down the network.

On a related note I tested this on a different platform - been mostly testing on VMware and an Intel Atom board. This evening I tested it on an embedded x86 Soekris board (no vga out) and it worked great. Not sure if the serial console works (didn't seem to at first blush), but it wasn't required with the other pieces the script enables as ssh was accessible out of the box.

Updated file in the first post - only the headless_patch file was updated.

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#14 Post by ldolse »

All of the changes discussed here are now integrated into mpdPup, if you want to see it in action. I did discover one last important item for anyone looking to duplicate this.

I added a setup wizard which launches from .ssh/rc for the first boot - the version in the Pet file in the first post only launches a warning about changing the password, but in my final version I wanted to launch a complete wizard. The end of the wizard asks to reboot the system.

The thing about anything running from the .ssh/rc script is that it isn't running from a tty, so the initial changes I added to rc.shutdown failed. Turns out that the SSH_TTY environment variable only exists for things initiated from SSH. If I tested the SSH_TTY variable from the monitor/keyboard while simultaneously running a separate SSH session the variable didn't even exist - it only existed in the context of the SSH session.

This enabled a further simplification of the check in rc.shutdown:

Code: Select all

if [[ $SSH_TTY ]]; then 
  CONSOLE="$SSH_TTY"
else
  CONSOLE="/dev/console"
fi
This will need some further refinement to get the serial console working - tested this and it definitely doesn't work. I think that will be easy enough to test for in rc.shutdown when that time comes.

I've attached my latest version of rc.shutdown here. If anyone is interested i can rebuild the pet in the first post.

I also discovered that there is some sort of security context issue with running from .ssh/rc with regard to passwords. I can't launch the 'passwd' command from inside that script successfully - it launches passwd but passwd won't prompt or change the password. Once the script is executed and you're in a normal tty then running the same script works fine... Not sure if there is a solution there, but I gave up on it and switched to just warning the user about the default password.
Attachments
rc.shutdown.zip
Shutdown script supporting SSH shutdowns.
(13.11 KiB) Downloaded 364 times

gcmartin

NAS sharing functionality would be needed if music stored

#15 Post by gcmartin »

ldolse wrote:No plans to include Samba, I'd rather store the files on a NAS, and I want the system focused on playing music, not transferring files.
I can see what @SBP is getting at with his reference to SAMBA. Its how we have been using our PCs over the past 17 years (Windows95).

If we have Multimedia files, we share it out over the LAN for other devices/users to use.

I also understand your focus of a audio-centric device. But, you may want to consider what @SBP offers in that, should anyone decide to use your Audio Headless system to also centrally store the media files, no user (especially newbies) should have to install ANY Linux subsystem (SAMBA is a Linux subsystem) to share those files with the plethora of media players that already exist in today's world and LANs.

SAMBA is NOT a MS connecti utility. In fact IBM, ATT, Novell was doing this initially with the International spec before Microsoft was given the spec by IBM. SAMBA is simply the agreement with Linus of how to best manage against ths spec for LAN services. And, like Puppy's author, another Australian partnered with Linus to produce this secure Linux subsystem.

I hope this is seen as providing helpful information about how we, users, have been using MS in the past for sharing multimedia (audio) files on the LAN for playback....no matter what other LAN devices we have which would want to use those audio files.

P.S. Have you seen your PM?

ldolse
Posts: 367
Joined: Fri 23 Oct 2009, 16:33

#16 Post by ldolse »

I don't have a problem with including Samba in general, though I certainly wouldn't recommend using it by default. The big reason I haven't included it to date is all of the wizard based configuration I've done is focused around having a NAS in the network already - if you have a NAS then there is no need for file server functions on the music server. I'm trying to avoid major features without also creating the wizards to back them up.

That said I hear it loud and clear that there are a number of people that would prefer to have their audio stored locally, and in that case I agree Samba would be really handy. I'll probably have one of the upcoming versions query the user as to whether they prefer their music library live directly on the server or whether it resides on the network, and then have the wizards configure things appropriately. For those who want it local Samba then becomes pretty important.

Just saw the PM, will respond separately.
Last edited by ldolse on Sat 17 Sep 2011, 05:52, edited 1 time in total.

gcmartin

Media Players on my LAN

#17 Post by gcmartin »

Here;s the most recent addition to my LAN ($99 Logitech). When it started up, it automatically went around my LAN and connected to media files everywhere it could find. I, now, can do all sorts of entertainmetn things from just this device.

There are others on my LAN as well from Netgear, Dlink, and Cisco (Linksys). The kids today, bring their Playstations over and everyone knows about the Multimedia aspects of these. Don't forget, too, that cell phones can access your media files over your LAN

I am not promoting SAMBA. It is a 20 year old Linux subsystem. We are just covering the MPDPUP Music and sharing it over the LAN, as well. This isn't even about SAMBA. This IS about Linux. The version when used with Linux that follows the International standard (RFCxxxx) for SMB sharing over the LAN. That's not SAMBA .... "something strange and different"....That's Linux with the subsystem for sharing called SAMBA. And, its not MS or HP or Citrix or IBM or Novell or ...Its Linux with a subsystem (SAMBA) for sharing and participation over your LAN/VPN

Hope this helps us look at what is happening in our homes and offices as our future is being grown by all of these new devices that are showing up on our LANs. It not about how I feel. Its what's happening for all of us.

Hope this helps.
Edited: 3rd paragraph's readability.
Last edited by gcmartin on Fri 16 Sep 2011, 21:03, edited 1 time in total.

nooby
Posts: 10369
Joined: Sun 29 Jun 2008, 19:05
Location: SwedenEurope

#18 Post by nooby »

I've seen this title now since Mon Sep 05

Creating a truly headless pup derivative
I have tried to get in what way it is headless and what that refers to
but I did not want to disturb the flow of exchanges in the trhead but
now when it is solved I dare to ask.

What on earh is a a truly headless pup derivative.

What is in the head? Is "head" another word for "Pinboard".

What does it lack when it is headless?
I use Google Search on Puppy Forum
not an ideal solution though

gcmartin

#19 Post by gcmartin »

Nooby wrote:What does it lack when it is headless?
Headless, typically, refers to a "box" (like a PC) that does not have monitor, keyboard, or mouse. It usually provides some useful service that is generally available to other devices that may exist on your local network.

A very good example
Your internet router

Hope this helps

nooby
Posts: 10369
Joined: Sun 29 Jun 2008, 19:05
Location: SwedenEurope

#20 Post by nooby »

You are a kind person indeed. Oh so that is how it is to be without the head. I would never have known unless you told me.

I thought them meant without Pinboard or X.

So it just starts and have a script that tells it what programs to run. Okay!

Yes the router maybe is almost like a computer. Has its own html homepage and can be programmed? to do different things. Set up rules and so on.

Them could build a whole computer into a Modem Router case :)
I use Google Search on Puppy Forum
not an ideal solution though

Post Reply