Sample Code & Cluster User's Guide

Below are links to some simple parallel programs, which serve as examples of the basic techniques.

Sample Parallel Programs

All three programs solve the same problem, namely, to repeatedly "smooth" or "filter" an arbitrarily-specified initial 3-dimensional array. The heart of each program is to repeatedly perform the operation:

array_out(i,j,k) = (1. - 6.*weight)*array_in(i,j,k) + weight*(
          array_in(i-1,j,k) + array_in(i+1,j,k) +
          array_in(i,j-1,k) + array_in(i,j+1,k) +
          array_in(i,j,k-1) + array_in(i,j,k+1) )

 

Some points to note here are:

1. Dependency problems are avoided by using a separate "array_out" instead of directly over-writing "array_in".
2. Over a finte 3-d array, special treatment is needed at the 6 faces, 12 edges and 8 corners, and the "brute force" approach is to treat each of these separately.
3. When the "global" 3-d domain is divided into discrete sub-domains for parallel processing, the (i+/-1, j+/-1,k+/-1) points can "belong" do a different sub-domain from the base-point (i,j,k). Points needed by one sub-domain, but "belonging" to another, form a set of "shadow-zones" or "halos" around each sub-domain. Since each sub-domain is controlled by a separate program (or MPI process), "shadow-zone" values need to be provided by communication or "message-passing" between each sub-domain.
4. The concept of a "shadow zone" also provides a more elegant way to handle the faces, edges and corners of the global 3-d domain: array_in is simply extended by another 2 points in each direction, and the original boundary values simply copied to the new boundary points. This extended "numerical" domain means that points (i+/-1,j+/-1,k+/-1) can be handled at the boundaries of the "physical" domain in exactly the same way as interior points.

Here are three sample programs to demonstrate each of the approaches mentioned above:

1. A Fortran 90 program, to do repeated 3-d array smoothing using OpenMP ("brute-force" method).
2. A Fortran 90 program, to do repeated 3-d array smoothing using OpenMP (condensed "shadow-zone" method).
3. A Fortran 90 program, to do repeated 3-d array smoothing using MPI.

Finally, here is a rudimentary User's Guide to Working on Clusters.