6.6) ARC2 Architecture And SGE Parallel Environments

Computer clusters like ARC2, incorporate both shared and distributed memory architectures. The individual nodes on ARC2 have a shared memory architecture, i.e. the 16-cores on a node all have access to 32GB of memory on the machine. This means that shared memory programming models can be used when making use of single node. The 16-core nodes are all connected to each other via a high speed network, and so form a distributed memory machine where use of message passing between nodes is necessary when running jobs over multiple nodes.

To accommodate these two modes of parallelism, the batch queue system has two main parallel environments configured. To specify the appropriate environment SGE must be passed the option –pe, short for parallel environment, followed by the one of the flags:

• smp np : to be used to request np cores for shared memory parallel programs. It will assign all the requested cores on a single node,. Thus, it can be used to request up to a maximum of 16 cores on ARC2, which is the number of cores on a single node.
• ib np : to be used to request np cores for distributed memory parallel programs, which will make use of message passing via the InfiniBand network.