MPICH: Pick Your Paradigm
From Debian Clusters
This is part two of a multi-part tutorial on installing and configuring MPICH2. The full tutorial includes
- Installing MPICH
- MPICH: Pick Your Paradigm
- MPICH with Torque Functionality OR MPICH without Torque Functionality
Contents |
With or Without Torque Functionality?
MPICH has the flexibility to be run a number of different ways. Specifically, I'll be covering how to run MPICH with the resource manager/scheduler Torque as well as how to run MPICH without a scheduler. If you know which you'll be using, continue on. Otherwise, see below.
Overview
Most clusters will need a scheduler/queuer. The rare exceptions may be a single-user cluster. If you're setting up a cluster solely for yourself, and you want to monitor when your jobs are running rather than having your jobs be scheduled for you, you might not need one. (Still, you'd be saving yourself a lot of monitoring overhead if you install one, so I'd still recommend it!)
MPICH can take advantage of this with the installation of a special mpiexec binary.
Torque-Aware Mpiexec
When running an mpi job, this special mpiexec will take of all the MPI setup and teardown. Users will simply specify how many processes they want, and by communicating with Torque to find out how many each alloted node has, mpiexec will automatically run the nodes allocated for a job.
However, this special version of mpiexec cannot be run outside of a Torque job submission. In other words, users will not be able to test their code without submitting it to Torque, and you won't be able to test any software running over MPICH until you've installed Torque. (This can be overcome by only installing the special binary on the worker nodes, and will be covered as part of the tutorial.)
I highly recommend taking this route.
Stand Alone MPICH
Without Torque (or a different scheduler), the above option is not possible.
Each user will need a special .mpd.conf file, and /etc/mpd.conf will need to be set up on each of the worker nodes. Then an "mpd ring," a collection of MPI daemons communicating with each other, is set up across the cluster.
If Torque is installed, it will not be able to enforce which machines users access with any MPI programs; hopefully, users will "play nice" and only use their allocated machines, but this may not be the case. Alternatively, users can start their own mpd rings within Torque submission scripts, but problems will arise if a node with multiple processors is scheduled to run two MPI jobs from the same user, since only one mpd per user can be started per machine.
I recommend taking this route only if you do not have Torque installed.
