Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 21 Nov 2014, 20:27
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
Puppy 245mpi cluster made easy
Moderators: Flash, JohnMurga
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 5 of 5 Posts_count   Goto page: Previous 1, 2, 3, 4, 5
Author Message
Master_wrong

Joined: 19 Mar 2008
Posts: 456

PostPosted: Fri 18 Feb 2011, 10:42    Post_subject:  

@gcmartin
Quote:

Thanks for clarifying everything you've done thru-out this Thread. I have followed you work religiously. Very clear objectives, approaches, conclusion.

I have one complex request.


* Would you setup your LiveCd to automatically start the PXE environment when it boots without user intervention.
* Would you post the LiveCD/LiveDVD that would boot the Cluster Master starting the PXE environment with those files necessary for the cluster nodes to PXE from, please.

This would make an automatic (another word for dummies like me) cluster where all I do is boot "Live" and start nodes, wthout ever typing or clicking on anything, to begin and run my cluster.

You may want to start a new Thread announcing your Multi PC "ClusterPUP".
See my PM to you.


well i can put netboot folder into root which will be saved during remaster, but the iso size will be enormously 600+M Confused

what's important is that now it can boot headless pc. but you still need to click

@Aitch Smile
Quote:

Anyone doing video/high intensity graphics/blender will benefit from this, however, can compiling be speeded up on a cluster, it is common to hear it taking 10/12 hrs?


here i test mpir...
it seems good

Quote:

# mpirun -np 2 ~/mpir -s 101 -e 1000 'blender -b in.blend -s %start -e %end -a'
sh: blender: command not found
sh: blender: command not found
^C

# mpirun -np 2 ~/mpir -s 101 -e 1000 'dvdauthor -b in.blend -s %start -e %end -a'
DVDAuthor::dvdauthor, version 0.6.12-alpha.
Build options: gnugetopt magick iconv freetype
Send bugs to <dvdauthor-users@lists.sourceforge.net>

ERR: Must first specify -t, -m, or -x.
DVDAuthor::dvdauthor, version 0.6.12-alpha.
Build options: gnugetopt magick iconv freetype
Send bugs to <dvdauthor-users@lists.sourceforge.net>
^C


off course i didnt install blender and i dont know howto run dvdauthor... Laughing
this is called renderfarm i guess
Anyone willing to test ? Wink [/quote]

_________________
Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
Back to top
View user's profile Send_private_message 
Master_wrong

Joined: 19 Mar 2008
Posts: 456

PostPosted: Sun 20 Feb 2011, 00:56    Post_subject: testing mpich2-132  

I have test Mpich2-132 on 5.2.
.
i have install required fortran compile and install it also run few test.

it seem new mpich didnt use mpdboot anymore, but hydra and what it need is just list of ip...
so i might have to create new script again.
edit... false alarm i didn configure it properly
mpich2-132.jpg
 Description   
 Filesize   20.45 KB
 Viewed   1426 Time(s)

mpich2-132.jpg


_________________
Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Tue 01 Mar 2011, 03:38    Post_subject: remote FS use in Cluster  

Don't know if you are aware, but, SAMBA has been used om Cllusters for awhile. You may find this use a lot in cluster community. See here for starters. then you should join a SAMBA forum where a large community of user are available that will help.

See SAMBA cluster reference here and the Cluster support covers here.

Hope this helps

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
Master_wrong

Joined: 19 Mar 2008
Posts: 456

PostPosted: Thu 03 Mar 2011, 11:14    Post_subject:  

Thank gcmartin, it's more like database solution for cluster and i bet the DB stands for Database...

Quote:

Requirements
CTDB relies on a clustered filesystem being available and shared on all nodes that participate in the CTDB cluster. This filesystem must be mounted and available on all nodes in the CTDB cluster.

On top of this cluster filesystem, CTDB then provides clustered HA features so that data from the clustered filesystem can be exported through multiple nodes in the CTDB cluster using various services. Currently included with CTDB are the necessary hooks for Samba, NFS and ftp exports. Support for new service types can easily be added.



but sure very useful for some cluster application.

_________________
Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Sun 06 Mar 2011, 15:43    Post_subject: CTDB and ClusterPup  

If you are familiar with sharing files from one Windows PC with the other PCs on the LAN, then this allows ClusterPUP to become a PC that can also share on the LAN, same as all Windows PCs.

Thus, ClusterPUP can not only "mount" Windows share resources, but with CTDB, it can make available folders and files for LAN PCs to read and write to for the cluster.. Now, as a simple example, I can load data, for cluster management/applications, directly from somewhere else on my LAN, AND, I can run applications within my cluster where the resulting data can be viewed, used, manipulated, or managed by things on my LAN. All this done with only lifting a finger!

Think of it as a "secure" data sharing gateway for your cluster.See this and PDF reference at bottom
and this wiki

Hope this helps

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
Master_wrong

Joined: 19 Mar 2008
Posts: 456

PostPosted: Mon 07 Mar 2011, 21:24    Post_subject:  

Smile

great additional info,

btw unfs3 can be configured with cluster, which means client can read and run program which stored in server.

otherwise we can also use glusterfs and cnfs
http://www.gluster.com/community/documentation/index.php/Storage_Server_Installation_and_Configuration

http://clusternfs.sourceforge.net/Presentation.html

_________________
Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Wed 09 Mar 2011, 18:35    Post_subject: Agreed  

[quote="Master_wrong"]great additional info,

btw unfs3 can be configured with cluster, which means client can read and run program which stored in server.

Here's another note on cluster SAMBA use: NO Microsoft PC delivers with nfs or dfs. A simple user KNOWS how to use Network Neighborhood to find, read and write to LAN locations.

That's the primary reason any of us needs SAMBA. Remember, 99.44% of ALL of the worlds LANs have Microsoft PC users operating on their LANs.

This does NOT mean that Microsoft users can't install something or learn something new, it only means that I don't have to teach any Microsoft user anything about reading and writing information using the "out-of-box" Microsoft OS on their PCs. Thus, if you have ANY Linux PC with SAMBA, all systems "see" each other on the LAN without any changes in Microsoft.

Thats the only reason I bring this to our attention.

Hope this helps.

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
Master_wrong

Joined: 19 Mar 2008
Posts: 456

PostPosted: Sun 13 Mar 2011, 11:14    Post_subject:  

@ gcmartin...

Quote:

That's the primary reason any of us needs SAMBA. Remember, 99.44% of ALL of the worlds LANs have Microsoft PC users operating on their LANs.


sure for casual work which involve windows i agree but my design for cluster was designed to work entirely closed environment, so there is no windows attached.
but samba capable would be great addition anyway.

_________________
Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199

Edited_times_total
Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Wed 16 Mar 2011, 23:40    Post_subject:  

To EVERYONE
If you haven't seen it, "ClusterPUP - A Very Simple Cluster Implementation" is a great document put together by our very own Master_Wrong.

Great explanation, with diagrams and picturs, which makes it easy for all of us to implement our own High performance cluster! Any of us can, now, take advantage of this and easily get a cluster operational in less than a half-hour.

Thanks

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
Aitch


Joined: 04 Apr 2007
Posts: 6825
Location: Chatham, Kent, UK

PostPosted: Fri 10 Jun 2011, 21:30    Post_subject:  

Hi guys

It's been awhile since this thread got commented.....

I've just come across a new reason for clustering/parallel computing....the green agenda/energy saving


Quote:
"Parallel Computing Infrastructures (PCI) - computer clusters with many CPUs interconnected via a dedicated network - constitute the preferred option for many Institutions requiring massive computational power. PCI typically run 24h per day, even when not all CPUs are being requested. This leads to unnecessary energy expenditure and associated CO2 emissions. Here we offer a solution saving up to 90% of PCI’s energy consumption during idle time in existing (and future) PCI. It is provided in the form of a free, open source software tool (we call it SPIRIT) which activates-on-demand requested CPUs, leaving all others in a shutdown state. Given the widespread adoption of PCI across the globe, we anticipate significant reductions in CO2 emissions upon implementation of this protocol."


http://www.ciul.ul.pt/~ATP/SPIRIT/


gcmartin/Master_wrong

Any comment as to a Puppy use for this?

Can the Sun Grid Engine implementation be easily adapted to ClusterPup, or other linux clusters?

Aitch Smile
Back to top
View user's profile Send_private_message 
drspastic


Joined: 29 Aug 2008
Posts: 90
Location: ruse, bulgaria

PostPosted: Sat 11 Jun 2011, 17:06    Post_subject:  

i think one day we will have a desktop icon in puppy called CLUSTER and when it is clicked it connects automaticly with every other puppypc similarly activated via network and internet globally sharing ram, storage, and cpu cycles. the microsoft buster finally.
please tell, can a cluster run without a single master? can slaves load balance each other? can the greater cluster run the master on a vm?
and most importantly can a cluster work all its power to run a massive virtual machine that users can log into as if on a normal desktop pc and run normal apps without the need for special adaptations?
maybe a massive puppy cluster vm could run xp at a reasonable speed. Wink
Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4436
Location: Earth

PostPosted: Sun 12 Jun 2011, 17:07    Post_subject:  

Aitch wrote:
Hi guys Can the Sun Grid Engine implementation be easily adapted to ClusterPup ... ?
Aitch Smile
Thanks for this post. I can see application; not just with ClusterPUP, but in additional uses....say for PXE. The model that they present runs on a cluster much like you see in the ClusterPup document.

It appears that the applications they offer, employs 20 year old technology to "wake up" a powered off PC to go to work. And, thusly, when no work exist, to go away. Thus, when workloads are low/non-existent, no electricity is used within the complex.

drspastic wrote:
... most importantly can a cluster work all its power to run a massive virtual machine that users can log into as if on a normal desktop pc and run normal apps without the need for special adaptations? ...
What you seem to be asking for is called a "Terminal server". Citrix, MS, IBM have already done this for Business communities. Google has just implemented there ChromeBook technologies which is an extended form of what you ask. Its happening (for awhile now in some cases) all around us, while the term "cluster" is not in your face.

Since many people don't know what a Terminal Server is, go to this thread and click on its enclosed link. You will be allowed to use a Linux version of a Terminal server.....Free for your evaluation as you desire with 500MB of personal storage.

For Everyone
ClusterPUP is a mechanism that all of us can implement within our local spaces using Puppy Linux. See the official "ClusterPUP - A Very Simple Cluster Implementation" Manual authored by Master_Wrong. Its easy to follow, too.

Take 2-3 PCs and the manual, and do this yourself. You'll be impressed with its power.

hope this helps.
P.S. I think the current and OFFICIAL CLUSTERPUP SITE is here.

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
Display_posts:   Sort by:   
Page 5 of 5 Posts_count   Goto page: Previous 1, 2, 3, 4, 5
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0954s ][ Queries: 12 (0.0067s) ][ GZIP on ]