Puppy for parallel computation using openMPI

A home for all kinds of Puppy related projects
Message
Author
Everitt
Posts: 331
Joined: Tue 19 Dec 2006, 21:59
Location: Leeds,UK or Birmingham, UK

Getting there!

#31 Post by Everitt »

Last night I burn the first disk of MatrixPuppy. I scrapped it after one boot, but then I burnt a slightly different one, and it almost worked. Just.

The server boots fine, and the nodes load up ok. The (Very) basic tests I've run have shown that code is being executed on all the nodes. I've written some VERY basic documentation, found in /readme.text (a typo, but I like it :) )
Things I know: They two binaries I wrote to poweroff and reboot the nodes are broken. I think I know why, but I haven't had any time to look into it.

If anyone is interested in trying it then I have a 142MB ISO that I'd be more than happy to upload for you, but I have no idea where I could host a file like that. I tried puppyiso@ftp.servage.net but it timed out at 99%, twice, so I guess that's not going to work.

msumner
Posts: 205
Joined: Fri 05 Jan 2007, 01:10
Location: Lincolnshire, England.

#32 Post by msumner »

Hi Everitt, I only just noticed this thread. I have fancied trying out something like this with a selection of old scrappers. I find people giving me some quite decent boxes, the latest two have 850 and 950 mhz durons with 384 and 512 mb ram. I would love to try puppy on these!

User avatar
HairyWill
Posts: 2928
Joined: Fri 26 May 2006, 23:29
Location: Southampton, UK

#33 Post by HairyWill »

Will
contribute: [url=http://www.puppylinux.org]community website[/url], [url=http://tinyurl.com/6c3nm6]screenshots[/url], [url=http://tinyurl.com/6j2gbz]puplets[/url], [url=http://tinyurl.com/57gykn]wiki[/url], [url=http://tinyurl.com/5dgr83]rss[/url]

Everitt
Posts: 331
Joined: Tue 19 Dec 2006, 21:59
Location: Leeds,UK or Birmingham, UK

#34 Post by Everitt »

msumner wrote:Hi Everitt, I only just noticed this thread. I have fancied trying out something like this with a selection of old scrappers. I find people giving me some quite decent boxes, the latest two have 850 and 950 mhz durons with 384 and 512 mb ram. I would love to try puppy on these!
Those machines are way out of my league. I'm testing this on PII's (350 + 450 oc'd to 510)
I got the memory useage for the nodes down to about 48MB earlier. Unfortunately I also noticed that I'd made a slight mistake, so you could only actually use parellel code. Add that to a few lesser bugs / updates I've been playing with and I don't see much point actually releasing alpha1. I'll try to get a fixed version out soon, but right now I'm getting odd errors. I know recompiling will fix it, but I think that might break something else. I need to do some reading around this.

Thanks HairyWill, that looks like it should do nicely. :D

Everitt
Posts: 331
Joined: Tue 19 Dec 2006, 21:59
Location: Leeds,UK or Birmingham, UK

#35 Post by Everitt »

Right, lets try this again.
Getting alpha2 ready has taken longer than I expected as I found a couple of pretty majour bugs that I had to work around. Hopefully though this econd alpha will last ong enough to actually get uploaded. :)
Changes include:
Rewrite of my mpi launcher script to include a lot more options, and to fix a couple of bugs.
Modification of library path varaibles to include the mpi libs (:D)

I'm going to update the readme, look into another possible bug, then try to upload. Fingers crossed. :)

Everitt
Posts: 331
Joined: Tue 19 Dec 2006, 21:59
Location: Leeds,UK or Birmingham, UK

#36 Post by Everitt »

Finally, I get something done!
The iso can be found here. It is called martixpup-1.0alpha2.iso. By taking out some of the features of stadard puppy that I see little need for I shrunk it enough that I could upload without it timing out, so if you'r favourite app is missing please don't be too nasty about it. :P

Usage:
Set up a network, consisting only of the computers you plan to use. Each machine should be linked to the network by it's eth0.
Pop the disk into one of the machines, and boot from it as per usual. This is the 'server'. It's the only machine you need really use.
The other computers must either be able to boot from a network, or have bootable floppies from http://www.etherboot.org As soon as the 'server' has loaded they can boot from the network, so power the on. After the usual puppy loading things you should be greeted by a message, and a password propt. You don't need to login, it's ready and waiting.
On the 'server'; read 'readme.text', and have a play.

I'm sorry I can't add more detailed notes, but I'm in the middle of packing to leave tomorrow.

User avatar
cb88
Posts: 1165
Joined: Mon 29 Jan 2007, 03:12
Location: USA
Contact:

#37 Post by cb88 »

what if this were implemented in the cyber cafes? just boot up matrixpup at night and run so folding apps.....

http://www.stanford.edu/group/pandegrou ... nload.html
Taking Puppy Linux to the limit of perfection. meanwhile try "puppy pfix=duct_tape" kernel parem eater.
X86: Sager NP6110 3630QM 16GB ram, Tyan Thunder 2 2x 300Mhz
Sun: SS2 , LX , SS5 , SS10 , SS20 ,Ultra 1, Ultra 10 , T2000
Mac: Platinum Plus, SE/30

mcewanw
Posts: 3169
Joined: Thu 16 Aug 2007, 10:48
Contact:

supercybercafe

#38 Post by mcewanw »

cb88 wrote:what if this were implemented in the cyber cafes?
ecomoney should get into this for his cybercafe. Just imagine having your own supercomputer to play on before anyone else comes in for a cup of coffee!

Actually if all puppy users could matrix their machines together, I could then hire out the "facilities" of this old wreck of a laptop to NASA or the local university, or maybe tell them why it is running so fast...

Alternatively, I'll just have to accept the offer of a free puppy linux virtual machine vnc over ssh account on some other puppy user's fast shiny new multi-processor machine. OK, so puppy only uses one core (does it?); surely I can use the other one?!!! haha

Fantastic effort on Matrix pup though Everitt. Impressive.

macadavy
Posts: 213
Joined: Mon 12 Jun 2006, 07:43
Location: Cascadia's Attic, eh?

Puppy Matrix

#39 Post by macadavy »

Everitt: Have you looked at CHAOS?
http://tinyurl.com/26n4as
mcewanw: A global Puppy matrix free & open source supercomputer?You've been nipping on the single malt again, n'est pas? But count me in!
btw- whisky: the Scots do it, the Irish do it, but who knew the Welsh do it, too?! http://www.welsh-whisky.co.uk/index.html
:D
[i]Welcome to my weird, wild, wonderful, wired world![/i]

Everitt
Posts: 331
Joined: Tue 19 Dec 2006, 21:59
Location: Leeds,UK or Birmingham, UK

#40 Post by Everitt »

This project could be addapted for folding pretty easily. Just remove the stuff concerning openMPI and add in the folding client. IIRC the 'clustering' is handeled remotely, so all you'd need to have is the PXE server to boot the nodes, and have the client auto run. Should be pretty easy I'd have thought...

As for Chaos? yeah, that's what you get if you do this properly. :D
This was just a bit of fun for me really, those guys actually know what they're doing!

Now Welsh whiskey, that does sound tempting. It's about £37 a bottle though! Damn my student budget and expensive taste!

User avatar
NickBiker
Posts: 155
Joined: Wed 01 Mar 2006, 13:06
Location: Guildford, UK
Contact:

Cluster Computing Using Puppy

#41 Post by NickBiker »

Hi

Hi

I am glad that others are thinking along the same lines as I have been - building a cluster of computers using Puppy.

I am a bit of an "old hand" at Puppy!

1. I am just re-compiling the Kernel etc for SMP (multi-core) support (see http://www.murga-linux.com/puppy/viewtopic.php?t=34186
2. Then re-compile GCC to support parallel code using OpenMP
3. Then use OpenMPI (was LAM-MPI) for cluster coding (see http://www.lam-mpi.org/)

I agree that there needs to be a mother unit ("Neo"), that the other clusters can then PXE boot from (I have done this bit already!). These "compute" clusters can be a barebones puppy, since they will just be used for number crunching.

I thought I would write a nice friendly script to set everything up, automatically sort out 'Neo' and the compute nodes, etc. And install a number of demo's...

But I am supposed to be doing real work!
Nick

See my Puppy webcam Guide at:
www.moyo.me.uk/pupplinux

Everitt
Posts: 331
Joined: Tue 19 Dec 2006, 21:59
Location: Leeds,UK or Birmingham, UK

#42 Post by Everitt »

I'm afraid I find myself in very much a similar position. My computer time is now mainly spent trying to fix friends laptops, or work out how to analyse detector out put to work out how particles in a collider decayed (sample data I'm afraid, I'm not trusted with real data until next year). If there's anything I've done that I haven't uploaded that you'd like to see, I'd be more than happy to share, but I've had to pretty much give up on this. I think most everything I did is out of date due to the rapid progress puppy has made in the recent past anyway.

Post Reply