Beyond PXE, cloud PXE --- SOLVED

Under development: PCMCIA, wireless, etc.
Message
Author
Master_wrong
Posts: 452
Joined: Thu 20 Mar 2008, 01:48

Beyond PXE, cloud PXE --- SOLVED

#1 Post by Master_wrong »

after taking pxe lesson in
http://www.murga-linux.com/puppy/viewto ... 348#488348

I thought about cloud pxe and remember that tecnosaurus said that we need site or ftp that give us connection without password...

one way to do that is by portforwarding our computer as server...

i have one howto down here for ftp, but if we ned web server it is similiar:
http://www.murga-linux.com/puppy/viewto ... 153#342153


ok, what next
Last edited by Master_wrong on Sat 24 Sep 2011, 17:32, edited 2 times in total.
Cluster-Pup v.2-Puppy Beowulf Cluster
[url]http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199[/url]

gcmartin

#2 Post by gcmartin »

Seems I remember 2 tools that already do that. You may want to look over their work. "Emerge" and "Netboot Me"

I'm sure there are other web PXE tools that do this. But, I'm not sure how they accomplish it. I willing to help if I can.

Hope this helps

Master_wrong
Posts: 452
Joined: Thu 20 Mar 2008, 01:48

#3 Post by Master_wrong »

ok, there some info at least...

http://www.netboot.me/

thank you for your tip gcmartin.

** http://www.murga-linux.com/puppy/viewtopic.php?t=64107
*** but I still need to know what to portforward ? -----> its web at port 80

let me try write a rough idea :

1. create humoungous initrd.gz and vmlinuz, using jamesbond's script see **

2 boot Puppy @192.168.2.130 local ----> as web server, put above files accordingly

3. portforward 192.168.2.130 -----> xxx.xxx.xxx.xxx internet address (see *** above)

4. using DynDNS.org or similiar (no-ip.com), lock internet address into ------> boot-puppy.webpages.net

5. create netboot.me cd which point to boot-puppy.webpages.net

6. boot client with netboot.me cd.

ok, please inform me if i makes mistake here
this project is possible :idea:


---------------------

Bravo for jamesbond giving more info:
http://www.murga-linux.com/puppy/viewtopic.php?t=65379

If you want to boot from a website (as your debian links alluded), it's very easy. All you need is the gpxe *) boot disk (cd/usb/floppy - choose whatever you want), and boot from it. When the computer boots, press Ctrl-B, and type:
Code:
dhcp net0
chain http://your-web-server/your-single-file-puppy

The webserver obviously can be anything, it can be ibiblio (if someone uploads the single-file-puppy there), or it can be local webserver as in my screenshot. Your single-file-puppy can be created in this way:
1. Use mknetboot.sh to get the humoungous initrd.gz and vmlinuz.
2. Use wraplinux to combine this humoungous initrd.gz and vmlinuz together.
There you have it!


*) I used gpxe because ipxe, at the time of writing, doesn't work for me. And it only provides iso download for CD, other media is not supported (ie you have to compile it yourself).
**) those mknbi tools referred in the debian links is now known as "wraplinux", which is what I use.
***) you don't even need to use wraplinux really. Only the mknetboot.sh step is required, because you can do this in gpxe as well
Code:
dhcp net0
kernel http://your-web-server/vmlinuz
initrd http://your-web-server/initrd.gz
boot
so it's webserver which need to be portforwarded ----> port 80
8)
Last edited by Master_wrong on Mon 18 Apr 2011, 17:23, edited 5 times in total.
Cluster-Pup v.2-Puppy Beowulf Cluster
[url]http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199[/url]

Master_wrong
Posts: 452
Joined: Thu 20 Mar 2008, 01:48

#4 Post by Master_wrong »

hi,

I just tested the gpxe and its working, my test was :
gpxe from http://rom-o-matic.net/gpxe/gpxe-1.0.1/ ... /build.php


after boot the gpxe cd from the iso, press ctrl-b and type:

dhcp net0
kernel http://192.168.2.130/vmlinuz
initrd http://192.168.2.130/initrd.gz
boot

as i already set up webserver at my linux mint so the location of the
files at linux-mint is at ---> /var/www
its working and puppy, btw we need to set permission so we can download the files.
also we need to press ctrl-b rather quick.

i hope we can edit the iso or something

*************************
i also test using

dhcp net0
kernel http://your-web-server/vmlinuz

as the setting in the http://www.dyndns.com/
it also work,

i didnt check
initrd http://your-web-server/initrd.gz

btw because i believe it will work too and i didnt want to download
100mb+ of my own files :).
Cluster-Pup v.2-Puppy Beowulf Cluster
[url]http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199[/url]

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#5 Post by jamesbond »

Master_wrong wrote:i hope we can edit the iso or something
If you compile gpxe / ipxe from source, you can specify the script that you want to include and automatically gets executed during boot.

gpxe scripting http://www.etherboot.org/wiki/scripting
ipxe scripting http://ipxe.org/scripting

Code: Select all

make bin/gpxe.iso EMBEDDED_SCRIPT=/path/to/your/script
Change gpxe.iso with ipxe.iso if you're using ipxe. Good luck.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

Master_wrong
Posts: 452
Joined: Thu 20 Mar 2008, 01:48

#6 Post by Master_wrong »

UPDATE : 12-april-2011
After i think about how silly to download my own files, i think about how cool to actually boot from internet....

so, today I successfully boot from web using iso i obtained from
http://rom-o-matic.net/gpxe/gpxe-1.0.1/ ... /build.php

----> below is sample embedded script add this text, see picture for test result
#begin sample

#!gpxe
dhcp net0
kernel http://myserver.dyndns.net/cloud/vmlinuz
initrd http://myserver.dyndns.net/cloud/initrd.gz
boot




#end of sample
#-----------------------
so thats great...

this great for computer that has no network pxe booting capability. i think.
because we can even use vbox

and put embedded script something like... (see screenshot-3)

#!gpxe
dhcp net0
kernel http://192.168.2.130/vmlinuz
initrd http://192.168.2.130/initrd.gz
boot


other than that its just cool to have ability to boot from internet.

thanks jamesbond

:D
Attachments
Screenshot-3.png
(6.79 KiB) Downloaded 1790 times
gpxe.png
(15.14 KiB) Downloaded 1540 times
Last edited by Master_wrong on Sat 24 Sep 2011, 17:24, edited 4 times in total.
Cluster-Pup v.2-Puppy Beowulf Cluster
[url]http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199[/url]

cyberdam
Posts: 19
Joined: Fri 03 Dec 2010, 22:52

#7 Post by cyberdam »

hello

I tested that and it works well (I included it. sfs in the initrd.gz).
By cons to operate pxe, I have not found a way to load or save the file on a server file_save.2fs nfs or tftp.

Have you any idea?

Master_wrong
Posts: 452
Joined: Thu 20 Mar 2008, 01:48

#8 Post by Master_wrong »

Thank for the feedback
I have not found a way to load or save the file on a server file_save.2fs nfs or tftp.
try
http://www.murga-linux.com/puppy/viewto ... 2&start=15

http://www.murga-linux.com/puppy/viewtopic.php?t=65232

ask jamesbond...
Cluster-Pup v.2-Puppy Beowulf Cluster
[url]http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199[/url]

cyberdam
Posts: 19
Joined: Fri 03 Dec 2010, 22:52

#9 Post by cyberdam »

I have already seen, but it does not work or I do not understand
but I finally managed to run with a save in the initrd.
I see init changed.
Attachments
init.rar
(20.93 KiB) Downloaded 640 times

Master_wrong
Posts: 452
Joined: Thu 20 Mar 2008, 01:48

#10 Post by Master_wrong »

@cyberdam...

so did you solve your problem ?
how did you solve it ?
Cluster-Pup v.2-Puppy Beowulf Cluster
[url]http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199[/url]

dlou99
Posts: 3
Joined: Wed 20 Apr 2011, 17:36

What do you guys think of using memdisk to PXE boot?

#11 Post by dlou99 »

What do you guys think about using memdisk to simulate a CDROM from an ISO image in order to PXE boot Puppy Linux?

We'd have to push the maintainers to add a couple of 6kB kernel modules and then run the 4kB 'memdiskfind' binary from the syslinux package at http://ftp.debian.org/pool/main/s/syslinux but after that, it would just be a simple matter of putting this in pxelinux.cfg

Code: Select all

LINUX memdisk
INITRD lupu-525.iso
APPEND iso
and it should PXE boot pretty much as if there were a Puppy CD in a physical CDROM drive.

Do you guys think that people would balk at adding 17kB (7kB compressed) to the initrd.gz?

More details here: http://www.murga-linux.com/puppy/viewtopic.php?t=67053[/code]

Master_wrong
Posts: 452
Joined: Thu 20 Mar 2008, 01:48

#12 Post by Master_wrong »

Do you guys think that people would balk at adding 17kB (7kB compressed) to the initrd.gz?
is it possible ?
even if it possible and didnt disturb anything...

I dont know howto
push the maintainers
Cluster-Pup v.2-Puppy Beowulf Cluster
[url]http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199[/url]

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#13 Post by jamesbond »

cyberdam wrote:I have already seen, but it does not work or I do not understand
but I finally managed to run with a save in the initrd.
I see init changed.
1. A pupsave inside initrd is gone once the computer is rebooted. Might as well run from RAM then.
2. The idea of loading pup.sfs from NBD/CIFS/SSHFS/whatever is so that you don't have to load the entire pup.sfs into RAM (which is the case if pup.sfs is embedded into initrd). Thus, more RAM available for the apps with trade-off that apps launch becomes slower (because it has to access the network share).
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

Re: What do you guys think of using memdisk to PXE boot?

#14 Post by jamesbond »

dlou99 wrote:What do you guys think about using memdisk to simulate a CDROM from an ISO image in order to PXE boot Puppy Linux?

We'd have to push the maintainers to add a couple of 6kB kernel modules and then run the 4kB 'memdiskfind' binary from the syslinux package at http://ftp.debian.org/pool/main/s/syslinux but after that, it would just be a simple matter of putting this in pxelinux.cfg

Code: Select all

LINUX memdisk
INITRD lupu-525.iso
APPEND iso
and it should PXE boot pretty much as if there were a Puppy CD in a physical CDROM drive.

Do you guys think that people would balk at adding 17kB (7kB compressed) to the initrd.gz?

More details here: http://www.murga-linux.com/puppy/viewtopic.php?t=67053[/code]
Sounds interesting. Fatdog64 already has the required modules, all that is needed is to move these modules from pup.sfs to initrd. Fatdog also has memdiskfind, but currently it's a dynamically linked, so it has to be recompiled with klibc or dietlibc to make it standalone so that it can be put into initrd (or if a static memdiskfind is already available, just use it). Then all we need is to patch /init with code to load the iso from MTD device instead of /dev/sr0.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

dlou99
Posts: 3
Joined: Wed 20 Apr 2011, 17:36

#15 Post by dlou99 »

jamesbond wrote:
cyberdam wrote: 2. The idea of loading pup.sfs from NBD/CIFS/SSHFS/whatever is so that you don't have to load the entire pup.sfs into RAM (which is the case if pup.sfs is embedded into initrd). Thus, more RAM available for the apps with trade-off that apps launch becomes slower (because it has to access the network share).
The downside is that you have to have a bloatload of network drivers in the initrd and either figure out which to load or go and load all of them.

User avatar
drspastic
Posts: 90
Joined: Fri 29 Aug 2008, 09:03
Location: ruse, bulgaria

still though....

#16 Post by drspastic »

the netbootme.usb image seems to boot on any old tosh and works well. but no puppy on the menu still.
can a few more of us lobby the guys on the netbootme forum to get some puplets added to the menu as standard. they have tinycore, and the very slick slitaz, but we just miss the dog.

c7jaff
Posts: 13
Joined: Mon 07 Feb 2011, 19:29

#17 Post by c7jaff »

This worked for me. It was much easier than PXE and I could do it in a sitting and booted to CD.

It only worked on the PC I made it with though. I used the rom o matic and chose ISO which I burned to CD. I then put the 2 needed files for each OS on my web site. The pup files were run through mknetboot.sh. I grabbed the tinycore files as they were and they loaded the quickest. netboot.me has a good iso for that. I also used the following code (for puppy 511) in the last rom o matic text area as mentioned in previous posts:

#!gpxe
dhcp net0
kernel http://mysite/vmlinuz
initrd http://mysite/initrd.gz
boot

It works but takes 60-90 minutes for my largest puppy file to load into RAM. The "initrd.gz" is 177 MB and that's probably why...but I still don't know what the bottleneck is. Is it a limitation of my bios or gpxe? Does anyone know? I have a 100 Mbps network. Downloading or copying that file only takes less than a minute in Windows or Linux when not using gpxe.

The tinycore files are these sizes and only take a couple of minutes to load:

tinycore.gz (8.9 MB)
bzImage (1.9 MB)

gcmartin

#18 Post by gcmartin »

c7jaff wrote:This worked for me. It was much easier than PXE and I could do it in a sitting and booted to CD.

It only worked on the PC I made it with though. I used the rom o matic and chose ISO which I burned to CD. I then put the 2 needed files for each OS on my web site. The pup files were run through mknetboot.sh. I grabbed the tinycore files as they were and they loaded the quickest. netboot.me has a good iso for that. I also used the following code (for puppy 511) in the last rom o matic text area as mentioned in previous posts:

#!gpxe
dhcp net0
kernel http://mysite/vmlinuz
initrd http://mysite/initrd.gz
boot

It works but takes 60-90 minutes for my largest puppy file to load into RAM. The "initrd.gz" is 177 MB and that's probably why...but I still don't know what the bottleneck is. Is it a limitation of my bios or gpxe? Does anyone know? I have a 100 Mbps network. Downloading or copying that file only takes less than a minute in Windows or Linux when not using gpxe.

The tinycore files are these sizes and only take a couple of minutes to load:

tinycore.gz (8.9 MB)
bzImage (1.9 MB)
@c7jaff, what you've done looks great. Its, yet another, way for any of us to setup a facility for a local PC to boot using PXE technology.

If I can contrast your method with the current Netboot method, current Netboot does NOT require anything on the local PC for it to boot, whereas, your method requires a "minimal starter on that local PC for it to boot.

Your question
I notice you saying 60-90 minutes...you mean 60-90 seconds, right? Even with the largest OSs I have worked with, it doesn't take much longer than booting from a CDROM (worst case 1-3 minutes with progress showing on the local PC's screen). So, I am assuming you do mean seconds, RIGHT?

If I understand your question, its asking WHY is netbooting technology so slow in comparison to say, "copying a file from one place on the network to another"? I don't know the answer to that, but it is an interesting question.

I have been using netboot technology for about 15 years, on various platforms, and I have alway told everyone that "When you come to work, turn on your PC, and go get your coffee. When you get back, your PC will be ready for work." Seems to still apply, for me, no matter Puppy/Linux/Microsoft/AIX/etc.

THANKS for showing us your structure for "support and booting" from a webpage.

Hope this helps

c7jaff
Posts: 13
Joined: Mon 07 Feb 2011, 19:29

#19 Post by c7jaff »

You're welcome and your post was helpful.

It takes 60-90 minutes! I missed out moving the mouse the last few times the screensaver was to kick in so I don't know for sure. I was thrilled anyway when it worked. It may be by default accessing my on board NIC which has failed and only trickling data through. I also have a PCI NIC on the PC which works great but is probably not the default. The PCI NIC is what's used once I boot up and it gives me no problems. I think the latter has come up as anything from eth2 to eth6 on various Linux OS's. Haven't figured out why yet. That problem probably prevented me from getting my NFS persistent boot to work which finally worked using another server. I'll work on trying this gPXE HTTP boot with another PC.

BTW - to boot the same puppy files (using failed on board NIC PC as server & a laptop as client) via PXE it only takes 5 minutes or less.

What's the "current Netboot" method? Is it PXE using DHCP and TFTP? If not, what's the distinction?

gcmartin

#20 Post by gcmartin »

c7jaff wrote: ... What's the "current Netboot" method? Is it PXE using DHCP and TFTP? If not, what's the distinction?
Current method is simple. It is also described in the document we made for you in this thread.

Anyway, you can use ANY PC that has a LAN card as your PXE server.

For the machine you will boot:
  • To keep its use simple, it is REQUIRED to have a PC with its NIC on the MB.
  • And,
    • it is also REQUIRED to either use the PC's selection menu for telling it to boot using its NIC as the first device
    • OR you must enter the BIOS of the PC and tell it to boot using its NIC as the first device.
That's it. Its pretty simple.

If you need help with that process or have question on that process, post there and someone will help.

Hope this helps

Post Reply