Puppy Linux Discussion Forum

[Master_wrong]

-----------------
edit march-18-2011 -Howto tutorial
there is a complete installation manual available at "ClusterPUP - A Very Simple Cluster Implementation" -
https://docs.google.com/document/d/1jUKsxfkjwmU69vABOY1slSk5E9BmsobGaJ3nZj4vrKk/edit?hl=en&authkey=CNDM24ML&pli=1#

-----------------------------

edit feb 27-2011 -newest puplet here----> with pxe, auto detect and setting

http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199

--------------------


http://hotfile.com/dl/94559819/a465c13/puppy-cluster.zip.html

this is my first puplet....
all it does is simplify setting of mpich cluster... and made it as easy as 123...

Edit-feb-1-2011:
attached below is new config-script
1. improve key-less ssh
2. fix counting processor
3. nfs script (need to install nfs_common.tar.gz from http://murga-linux.com/puppy/viewtopic.php?t=45090
both server and client will be able to access /mnt/my. server need to mount sda1)

no more installing stuff into your desktop pc , al you need to have is cd and no need to setting ssh password-less and mpich which complicated for most ppl.
bottom line... no more headache for trying cluster computer....

see picture
1. basic setting, time date, hostname, nothing to edit here, my program will automatically set the hostname. press ok
2. Network wizard, select your network setting accordingly
3. If connection to network succed, a question will pop up, leave them if you are not on server pc
4. If you are on server pc then click yes and enter the list ip of all pc, put server ip on top, save and exit.
5. First time ssh will ask question, answer yes
6. Run mpd, compile program c, then run it, what was wrong ?... forgot to distribute the compiled program... use cluster-ssh-fileshare.sh
7. Program work after compiled program are distributed across cluster pc

I you want to know more about clustering....
http://www.phy.duke.edu/~rgb/Beowulf/beowulf_book/beowulf_book/index.html

also here is pdf to help using mpich2
http://www.fz-juelich.de/jsc/datapool/page/4224//install.pdf

Thank to gcmartin for the picture:
https://docs.google.com/drawings/edit?id=16jMd2gmt5YmmOlLWmb1q7fhufv4m0qKoKCZQJ348J54&hl=en&pli=1

have a nice day

[puppyiso]

Can you provide more detailed use manual?

Most importantly how to configure the network and the internet?

CD only or can be installed to USB?

John

[Master_wrong]

[Master_wrong]

I add interesting program....


flop.c ----> to count how many flop your cluster pc has to compare with those super computer.

now i can say that my puppy has more flop than yours

[gcmartin]

I run a LiveCD environment. So, for clustering several PCs to accomplish workload under a single-headed "Master" is an accomplishment for certain types of workloads. I applaud your accomplishment as this is significant.

I'm new to PC clustering. I have couple of questions? (assume a 3 PC running cluster)

• Is your work for 32bit or 64bit PCs?
• Does your current work require all of the cluster machines to be same?
• To dispatch work, how do the cluster machines know when to pitch-in (dynamic work allocation or static)?
• Assuming that the master has 2 LAN cards (1 to cluster switch-other to normal LAN)Can the master run a full Puppy desktop configuration as it handles workload management to those in the cluster?

This appears, to me, to be a very significant breakthrough in the Puppy world.
Thanks in advance

[gcmartin]

@Master ...

[Master_wrong]

Hi Gcmartin,



I was confused with my finding myself....

the first data with -np 1 -----> is using 1 processor only so there puppy-1 700 Mflop
2nd data -np 2 -----> 2 processor, because its dual core both are puppy-1 14000 Mflop, exactly double

so the funny stuff begun when other cluster join in

3rd data -np 3 -----> use 3 processor so it use puppy-1 and puppy-2 the drop to 590Mflop
4th data -np 4 -----> use 4 processor another drop to 400 Mflop, all processor on both computer are used

I assume because my destop(puppy-1) is newer than my laptop(puppy-2) this actually means that my laptop are too slow which hit the flop, but if we see the error then we see that clustering help minimize the rate of error.

[gcmartin]

Master...
I hope this makes sense. When I run an OS aware of multi-core processors, the hardware, under OS direction sometimes, is managing what one can consider to be "clustering".

When we move from local hardware clustering into "soft" clustering, we introduce a "new" bus that is used in communication of process controls, where it is significantly slower than the local multi-core bus. This should impact the behavior of some applications. So, application design is a factor depending upon what the objectives are to be.

In this case, we are seeing how "this particular program, only," behaves in your cluster. Another program with different design objectives in its design would be able to exploit you current cluster in a different fashion such that you would meet a different result. Further, there MAY be some cluster OS controls which, too, MAY affect /enhance program's performance in soft cluster.

Hope this helps.
P.S, My question on Puppy was to try to determine if you felt that a X-desktop console would contribute to the overall issues that would benefit cluster management, use, and performance.

[Master_wrong]

I try to compute how much flop does my laptop has....

it s just 200...
so imagine this pararel configuration while 2 other has 700 and my laptop has 200 so what happen is those faster processor just wait idle for the laptop to finish his work and dropping the flop count in the process.

it also makes senses too that
200x3 = 600 equal to the result of -np 3 , because workload is split by 3 processor
200x2 = 400 equal to the result of -np 4, because workload is split by 2 computer

btw my laptop seem just single processor, puppy see it as single processor and mpi treated it as single processor.

[whatupuppy]

Can you let me know your build process for puppy-cluster?

I'm waiting for a couple of computer and will test your puppy-cluster build.

Can not wait.

[Master_wrong]

[acrocosm]

Hmm so considering I have this properly set up and running on 3-4 machines, can I run blender to render a single image on all of them without having to configure anything else?

Or i need to run blender with the same scene on each of them and from the master machine I'll say: slave 1 render this layer, slave 2 render that layer and so on?

[Master_wrong]

you need a specially patched program.

but maybe i was wrong....


try.... http://svat.sourceforge.net/

they use blender...

[acrocosm]

Thanks for the link and of course thank you for your work!! I'll study that site and see what I can figure out.

Also I wanted to ask... is this vanilla luci 245 with a special sfs for clustering? Can I use the sfs on 5.2 or any other pupplet as long as the dependencies are satisfied?


Ultimately what I'd like to do is use the cannibalized spare machines I found on the streets or from friends to build a mini renderfarm for me and a friend. But I'm not so clear about distributed, grid, clustering and so on. I'd be obliged if you could clarify...

I'd like to run those machines diskless and if possible monitorless. Because I have 3-4 machines on the same room I don't see any reason in setting up a network where I run blender with the same scene and from one of those I tell machine2 render frames 1-100, machine3 render 101-200 etc. I can do that on each of them manually, they are next to another. Or I could use VNC to remotely control them and start the rendering.

What I'd like is all of them to act as one, shared memory, shared cpus. I don't expect shared graphics accelerators but it would be a nice touch. Is that possible? Is that a beowulf? Can I do that with this pupplet?

Sorry if I'm such a noob ^_^ and thank you again for your work. Using super fast and light puppy for clustering is such a brilliant idea!

[acrocosm]

Ok I'm setting this up on 2 machines to check if blender can see all CPUs. I got something odd...