The time now is Tue 24 Apr 2018, 23:23
All times are UTC - 4 |
Author |
Message |
jpeps
Joined: 31 May 2008 Posts: 3217
|
Posted: Fri 28 Jan 2011, 23:22 Post subject:
|
|
acrocosm wrote: | Master_Wrong I'm sorry, I have to clarify...
The original scripts included on the iso would show 2 cpu for the x40 on the server. Both of your later auto-config scripts worked fine. The server would see just one cpu on the x40 as expected
However, Code: | grep -c processor /proc/cpuinfo | still gives 2 as a result. |
grep -c ^processor
|
Back to top
|
|
 |
gcmartin
Joined: 14 Oct 2005 Posts: 6730 Location: Earth
|
Posted: Sat 29 Jan 2011, 13:20 Post subject:
Cluster applications |
|
IFF you are wondering how others have applied clusters, other than the one's I've mentioned earlier, look here for ways to use yours!
Hope this helps
_________________ Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Sat 29 Jan 2011, 15:01 Post subject:
|
|
@gcmartin
Quote: | P.S. It occurs that if we could replicate something like this in a Puppy Cluster, we could run every PUP, every PET, every SFS, and everything in the cluster. Now, hmmmm...this is some real food for those Workers to feed on from the Master. Big Dogs versus Small Dogs rules will still apply, though. But, the work will be spread in a very interesting manner. |
I dont think that using cluster will benefit all program, such as office suite etc they dont need that much processing power...
i believe that those computer that waiting idle in screensaver mode would be more useful if used as cluster. but not all program need cluster processing power.
_________________ Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
|
Back to top
|
|
 |
jpeps
Joined: 31 May 2008 Posts: 3217
|
Posted: Sat 29 Jan 2011, 16:12 Post subject:
|
|
Master_wrong wrote: |
I dont think that using cluster will benefit all program, such as office suite etc they dont need that much processing power |
Seems like office suite, etc., would need unique software to make use of clustering (i.e, that can run parallel tasks); doesn't make much sense for basic apps.
|
Back to top
|
|
 |
gcmartin
Joined: 14 Oct 2005 Posts: 6730 Location: Earth
|
Posted: Sat 29 Jan 2011, 16:59 Post subject:
Mistaken understanding |
|
Master_wrong wrote: | Quote: | P.S. It occurs that if we could replicate something like this in a Puppy Cluster ... . | I dont think that using cluster will benefit all program, such as office suite etc they dont need that much processing power...
i believe that those computer that waiting idle in screensaver mode would be more useful if used as cluster. but not all program need cluster processing power. | Thanks Master_wrong (and @jpeps too). I agree with you that today's Office Suite is NOT an application for clustering (and I don't believe I suggested it either).
The example I provided was one where someone has found a way to dispatch VMs as work in a cluster. That's novel, if it works as they suggest and is a greater way to throw workers at workload. I only threw that our way in Puppyland to inspire ideas for what can be accomplished in a Puppy cluster.
Hope this helps
_________________ Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Sat 29 Jan 2011, 23:14 Post subject:
|
|
Quote: | Thanks Master_wrong (and @jpeps too). I agree with you that today's Office Suite is NOT an application for clustering (and I don't believe I suggested it either).
|
well...
you did said
Quote: | we could run every PUP, every PET, every SFS, | ,
which i reply
Quote: | I dont think that using cluster will benefit all program, such as office suite etc |
sorry if i make it sound you were suggesting it but office suite does packaged as pet and sfs.
btw if we see from this picture
http://pve.proxmox.com/wiki/File:Pve-virtual-machines.png
its not virtual, it seems real pc, however they they run virtual macine on each pc node.
so its not that we run 1 virtual machine on server that use cluster processors.
I already find software that monitor each node, but its not as polished as theirs.
Quote: |
Description:
------------
Gossimon is a gossip based distributed monitoring system for a cluster of
Linux nodes. Each node in the cluster periodically send information about
itself and others to a randomly selected node. This way each node constantly
receive information about cluster nodes. This information is locally
maintained (constantly updated) by each node and can be used by various
clients.
|
_________________ Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Sun 30 Jan 2011, 07:31 Post subject:
|
|
This is my code to enable nfs,
192.168.2.130 is server
192.168.2.118 is client
http://murga-linux.com/puppy/viewtopic.php?t=45090
we only need to install nfs_common.tar.gz from above site, in the end both server and client will be able to access /mnt/my.
(already included at the 1st post)
####NFS ---need nfs_common.tar.gz only
### change drive="sda1" to your mounted device
## please create ~/my-applications/clusterdata/ip-list which contain list of ip
#need passwordless ssh
drive="sda1"
mm=`head -n 1 ~/my-applications/clusterdata/ip-list`
m=`echo ${mm/ /}`
echo "M="$m
rm /etc/exports
echo "/mnt/$drive *(rw,no_root_squash,no_subtree_check,sync)" >>/etc/exports
modprobe nfsd
/etc/rc.d/rc.nfsd start
x="mount-FULL -t nfs $m:/mnt/$drive /mnt/my "
echo "x="$x
for i in `cut --delimiter=: -f1 ~/my-applications/clusterdata/ip-list`;do ssh $i "mkdir /mnt/my"; ssh $i "modprobe nfs";ssh $i "/etc/rc.d/rc.portmap start";ssh $i $x;done
####NFS
_________________ Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
Last edited by Master_wrong on Tue 01 Feb 2011, 07:14; edited 1 time in total
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Mon 31 Jan 2011, 13:04 Post subject:
|
|
Although i did not agree gcmartin vision about possibility running all program using cluster, i make some research and put them here, i hope someone could use them...
from http://www.open-mpi.org/faq/?category=running
Quote: |
12. Can I run non-MPI programs with mpirun / mpiexec?
Yes.
Indeed, Open MPI's mpirun and mpiexec are actually synonyms for our underlying launcher named orterun (i.e., the Open Run-Time Environment layer in Open MPI, or ORTE). So you can use mpirun and mpiexec to launch any application. For example:
shell$ mpirun -np 2 --host a,b uptime
This will launch a copy of the unix command uptime on the hosts a and b.
Other questions in the FAQ section deal with the specifics of the mpirun command line interface; suffice it to say that it works equally well for MPI and non-MPI applications.
|
i know its open-mpi, but it s the same implementation, we can try...
we also have mpir..
or we can just run ssh to each computer and run the program
like these
# for i in `cut --delimiter=: -f1 ~/mpd.hosts`;do ssh $i "ls /mnt/my/$i";done
if we need to do blender, we can use nfs then create folder on master node with the name of the ip.
then set to run blender using the folder.
so the folder on master node will will look like these
/mnt/my/192.168.2.130/
/mnt/my/192.168.2.118/
/mnt/my/192.168.2.122/
the picture that needed to blended and the result will be saved into that folder, so each folder will contain different picture.
from http://wiki.blender.org/index.php/Doc:Manual/Render/Command_Line_Options
Quote: | In some situations we want to increase the render speed, access blender remotely to render something or build scripts that use blender command line.
One advantage of using command line is that we don't need the X server (in case of Linux) and as a consequence we can render remotely by SSH or telnet.
|
next...
run python program on mpi
http://sourceforge.net/projects/mpy/files/mpych/0.1/mpi_wrap_mpych-0.1.linux-i686.tar.gz/download
_________________ Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
|
Back to top
|
|
 |
gcmartin
Joined: 14 Oct 2005 Posts: 6730 Location: Earth
|
Posted: Mon 31 Jan 2011, 13:14 Post subject:
Mistaken understanding |
|
Master_wrong wrote: | Quote: | Thanks Master_wrong (and @jpeps too). I agree with you that today's Office Suite is NOT an application for clustering (and I don't believe I suggested it either).
| well...
you did said Quote: | we could run every PUP, every PET, every SFS, | , which i reply Quote: | I dont think that using cluster will benefit all program, such as office suite etc | sorry if i make it sound you were suggesting it but office suite does packaged as pet and sfs.
btw if we see from this picture
http://pve.proxmox.com/wiki/File:Pve-virtual-machines.png
its not virtual, it seems real pc, however they they run virtual macine on each pc node. so its not that we run 1 virtual machine on server that use cluster processors.
I already find software that monitor each node, but its not as polished as theirs. | Yeah, I can see how this was misleading. Newspapers are constantly taking a quote out of context. When I made that quote (look at the whole passage), I was making it meaning to have whole OS's run and their services being delivered from the internal OS's. So, for example, if I dispatched a OS as a cluster application, whatever internal services that OS provides could be available via that OS....not the cluster directly...internal from the OS. In the case of Puppy (an OS), I would have services (PET/SFS) within it available.
On the item of PROXMOX, I agree without lifting the covers on how they achieved their implementation, it might not be a cluster; but it they are, in fact, clustering, then they have an implementation where we can draw upon for understanding and for future Puppy implementations.
Here's the kind of idea that one can draw: Physical Cluster with Virtual machines running with some virtual Machines actually being a cluster with other virtual machines with ... I think everyone can see where this leads to... you know physical to virtual to virtual to... alongside other "real" individual cluster applications being globbled up by physical and virtual cluster machines.
Anyway, you have made a very promising contribution to the Puppy community thus far. And I applaud your achievement!
Hope this helps
_________________ Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Mon 31 Jan 2011, 13:43 Post subject:
|
|
Quote: | Yeah, I can see how this was misleading. Newspapers are constantly taking a quote out of context. When I made that quote (look at the whole passage), I was making it meaning to have whole OS's run and their services being delivered from the internal OS's. So, for example, if I dispatched a OS as a cluster application, whatever internal services that OS provides could be available via that OS....not the cluster directly...internal from the OS. In the case of Puppy (an OS), I would have services (PET/SFS) within it available. |
i guess.. forgive me for being me.... ?
but i guess that would be too difficult to implement. then again i could be me ?
btw i just read this...
Quote: |
What is a beowulf cluster or Beowulf cluster vs Supercomputer
Beowulf cluster is essentially a number of commodity or of the
shelf PCs tied together using some sort of protocol. Developed
by NASA, Beowulf clusters gained much attention in the last few
years. Often at a fraction of a cost of a "real" supercomputer
a beowulf cluster is comparable (or even exceeds) the
computational power of a super computer. For instance the a
cluster made out of 320 ubiquitous P3-800 CPUs takes 34th place
on the TOP500 super computer list with peak performance of over
a teraflop!
|
ok has anyone have 320 p3 computer to test ?
_________________ Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
|
Back to top
|
|
 |
whatupuppy
Joined: 17 Jan 2011 Posts: 17
|
Posted: Wed 02 Feb 2011, 12:02 Post subject:
|
|
Update:
Well things did not go well with my friend this weekend.
So this have to be put on hold for now.
Also, failing hard drive on old laptops causing puppy to hang at Uncompressing linux..OK.booting Linux kernel.
Any suggestion beside buying a new harddrive is welcomed.
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Wed 02 Feb 2011, 20:59 Post subject:
|
|
I was about to suggest buying flashdisk....
so there is no cdrom, i guess ?
_________________ Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Wed 02 Feb 2011, 21:19 Post subject:
|
|
i spend last night testing like couple dozens of program....
it end up only couple of program that compiled successfully
here is 2 interesting program that might be useful,
Quote: | 1. tentakel :
Tentakel is a program for executing the same command on many hosts in parallel
using various remote methods. It can make use of several sets of hosts that are
defined in a configuration file as groups.
It also supports an interactive mode that can be used for repeated commands.
The author uses tentakel to simultaneously install software on many
workstations or ask and set parameters on a linux compute cluster. With the
power of format strings tentakel can also be used for monitoring purposes.
Supported remote methods are ssh(1) and rsh(1). Both must be configured to
allow for password-less logins. Password-protected keyfiles for ssh can be
used with ssh-agent(1).
A plugin mechanism allows users to implement their own remote methods in
addition to the builtin ones.
For more information on available options please refer to the manpage
tentakel(1).
The project homepage is: http://tentakel.biskalar.de/ |
too bad i cannot get anything to work, i simply not smart enough to create config file, if anyone capable please let me know
Quote: |
2. unfs3-0.9.22.tar.gz :Great NFS for puppy, it capable of export and mount /root
INTRODUCTION
============
UNFS3 is a user-space implementation of the NFSv3 server
specification.
UNFS3 supports all NFSv3 procedures with the exception of the
READDIRPLUS procedure. It tries to provide as much information
to NFS clients as possible, within the limits possible from
user-space.
See the unfsd( manpage for restrictions imposed on NFS
operations (section RESTRICTIONS) and for possible races
with local file system activity (section BUGS).
It is not possible to export to netgroups or wildcard hostnames
via /etc/exports, all other addressing methods should work. The
following options are recognized in the exports file: ro, rw,
root_squash, all_squash, no_root_squash, no_all_squash. If
other options are present, they are ignored.
Cluster extensions compatible to the older ClusterNFS project
are supported when the source is configured with --enable-cluster.
|
all you need to do
1. create /etc/exports file: contain=====> /root (ro) --->for read only
open terminal at unfsd directrory
2. ./configure
3. make
4. make install
5. ./unfsd -s -d -p -n 4711 -m 4711 -e /etc/exports
6. at client : mkdir /mnt/try
7. copy unfsd,./configure,make,make install
8. mount -o port=4711,mountport=4711,mountvers=3,nfsvers=3,nolock,tcp 192.168.2.130:/root /mnt/try
Description |
|
Filesize |
25.94 KB |
Viewed |
1072 Time(s) |

|
Description |
|

Download |
Filename |
tentakel.zip |
Filesize |
42.15 KB |
Downloaded |
369 Time(s) |
Description |
|

Download |
Filename |
unfs3-0.9.22.tar.gz |
Filesize |
164.5 KB |
Downloaded |
389 Time(s) |
_________________ Cluster-Pup v.2-Puppy Beowulf Cluster
http://www.murga-linux.com/puppy/viewtopic.php?p=499199#499199
|
Back to top
|
|
 |
Master_wrong
Joined: 19 Mar 2008 Posts: 456
|
Posted: Fri 04 Feb 2011, 21:43 Post subject:
|
|
made pet of unfs3 at http://www.murga-linux.com/puppy/viewtopic.php?t=64491
btw pxe booting work using
http://www.murga-linux.com/puppy/viewtopic.php?t=64107
also other program tested
Quote: |
pconsole does not work through daemons. The pconsole.sh script opens up a
number of connections to each node of your cluster of workstations. Then,
the pconsole binary attaches to the tty devices that these windows are
using, and it copies the input you type to all open connections......
.......
By default, pconsole tries to use SSH to make connections. If you do not have
'ssh', it will try to use 'telnet'.
|
Description |
|

Download |
Filename |
pconsole-test.png |
Filesize |
32.67 KB |
Downloaded |
358 Time(s) |
Last edited by Master_wrong on Sun 06 Feb 2011, 02:12; edited 2 times in total
|
Back to top
|
|
 |
gcmartin
Joined: 14 Oct 2005 Posts: 6730 Location: Earth
|
Posted: Sat 05 Feb 2011, 13:58 Post subject:
What is a cluster |
|
@Master _Wrong has done all of us a very valuable service. I am showing this picture here because over the past couple months, many who come here have an inaccurate perception of a cluster.
[img]http://docs.google.com/drawings/edit?id=16jMd2gmt5YmmOlLWmb1q7fhufv4m0qKoKCZQJ348J54&hl=en[/img]
This picture is worth a thousand words, as, it shows where to place the monitor, keyboard, and mouse for the cluster. In a true cluster, the nodes/workers/slave PCs DO NOT HAVE A MONITOR, KEYBOARD, OR MOUSE. But they all work together on behalf of the workload within the cluster.
Hope this helps
Description |
|

Download |
Filename |
Beowulf.png |
Filesize |
86.79 KB |
Downloaded |
379 Time(s) |
_________________ Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile
Last edited by gcmartin on Sat 05 Feb 2011, 14:28; edited 1 time in total
|
Back to top
|
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|