MPI使用指南

系统安装了3种类型的MPI库及编译器,使用时务必使用SLRUM调度系统提交任务(如srun、sbatch、salloc等),并使用hpib分区(-p hpib)。3种MPI类型分别为:

Intel MPI

使用时添加相应的环境变量:
[simon@n0255 bin]$vim ~/.bash_profile
...
###for Intel 2016 compiler
export I_MPI_PMI_LIBRARY="/usr/lib64/libpmi.so"
source /home/software/intel/parallel_studio_xe_2016_update2/bin/compilervars.sh intel64
...

版本:
[simon@n0255 bin]$ which mpirun
/home/software/intel/parallel_studio_xe_2016_update2/compilers_and_libraries_2016.2.181/linux/mpi/intel64/bin/mpirun
[simon@n0255 bin]$ mpirun --version
Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160120 (build id: 14053)
Copyright (C) 2003-2016, Intel Corporation. All rights reserved.
[simon@n0255 bin]$

使用示例:
假设有如下 mpitest.c 程序,内容如下:

#include "mpi.h"
#include <stdio.h>
#include <string.h>
int main (int argc, char *argv[])
{
int i, rank, size, namelen;
char name[MPI_MAX_PROCESSOR_NAME];
MPI_Status stat;
MPI_Init (&argc, &argv);
MPI_Comm_size (MPI_COMM_WORLD, &size);
MPI_Comm_rank (MPI_COMM_WORLD, &rank);
MPI_Get_processor_name (name, &namelen);
if (rank == 0) {
printf ("Hello world: rank %d of %d running on %s\n", rank, size, name);
for (i = 1; i < size; i++) {
MPI_Recv (&rank, 1, MPI_INT, i, 1, MPI_COMM_WORLD, &stat);
MPI_Recv (&size, 1, MPI_INT, i, 1, MPI_COMM_WORLD, &stat);
MPI_Recv (&namelen, 1, MPI_INT, i, 1, MPI_COMM_WORLD, &stat);
MPI_Recv (name, namelen + 1, MPI_CHAR, i, 1, MPI_COMM_WORLD, &stat);
printf ("Hello world: rank %d of %d running on %s\n", rank, size, name);
}
} else {
MPI_Send (&rank, 1, MPI_INT, 0, 1, MPI_COMM_WORLD);
MPI_Send (&size, 1, MPI_INT, 0, 1, MPI_COMM_WORLD);
MPI_Send (&namelen, 1, MPI_INT, 0, 1, MPI_COMM_WORLD);
MPI_Send (name, namelen + 1, MPI_CHAR, 0, 1, MPI_COMM_WORLD);
}
MPI_Finalize ();
return (0);
}

使用mpicc进行编译,生成mpitest可执行文件:
[simon@n0255 ~]$ mpicc -o mpitest mpitest.c
[simon@n0255 ~]$ ls mpitest*
mpitest mpitest.c
[simon@n0255 ~]$

使用srun进行提交(注意,提交时,务必使用slurm调度系统,如srun、sbatch等,提交的分区为hpib,即使用-p hpib 参数):
[simon@n0255 ~]$ srun -p hpib -n 10 mpitest
Hello world: rank 0 of 10 running on n0317
Hello world: rank 1 of 10 running on n0317
Hello world: rank 2 of 10 running on n0317
Hello world: rank 3 of 10 running on n0317
Hello world: rank 4 of 10 running on n0317
Hello world: rank 5 of 10 running on n0317
Hello world: rank 6 of 10 running on n0317
Hello world: rank 7 of 10 running on n0317
Hello world: rank 8 of 10 running on n0317
Hello world: rank 9 of 10 running on n0317

说明:-n 参数后接并行任务的数量。

使用sbatch提交脚本如下(mpitest.sbatch):
#!/bin/bash
#SBATCH -n 10
#SBATCH -p hpib
srun -p hpib -n 10 mpitest

提交:
sbatch mpitest.sbatch

MPICH

使用时添加相应的环境变量:
[simon@n0255 bin]$vim ~/.bash_profile
...
###for mpich
export PATH=/home/software/mpich_3.2/bin:$PATH
export MANPATH=/home/software/mpich_3.2/share/man:$MANPATH
...

编译以及调用:
[simon@n0255 ~]$ which mpicc
/home/software/mpich_3.2/bin/mpicc
[simon@n0255 ~]$ mpicc -o mpitest mpitest.c
[simon@n0255 ~]$ srun --mpi=pmi2 -p hpib -n 10 mpitest
Hello world: rank 0 of 10 running on n0317
Hello world: rank 1 of 10 running on n0317
Hello world: rank 2 of 10 running on n0317
Hello world: rank 3 of 10 running on n0317
Hello world: rank 4 of 10 running on n0317
Hello world: rank 5 of 10 running on n0317
Hello world: rank 6 of 10 running on n0317
Hello world: rank 7 of 10 running on n0317
Hello world: rank 8 of 10 running on n0317
Hello world: rank 9 of 10 running on n0317
[simon@n0255 ~]$

说明:–mpi=pmi2 使用 pmi2进程管理接口 (pmi:Process Management Interface),当使用 mpich 时,此参数为必选项。

使用sbatch提交脚本如下(mpitest.sbatch):
#!/bin/bash
#SBATCH -n 10
#SBATCH -p hpib
srun --mpi=pmi2 -p hpib -n 10 mpitest

提交:
sbatch mpitest.sbatch

OpenMPI

使用时添加响应的环境变量:
###for openmpi
export PATH=/home/software/openmpi_1.10/bin:$PATH
export LD_LIBRARY_PATH=/home/software/openmpi_1.10/lib:$LD_LIBRARY_PATH

编译以及调用:
[simon@n0255 ~]$ which mpicc
/home/software/openmpi_1.10/bin/mpicc
[simon@n0255 ~]$ mpicc -o mpitest mpitest.c
[simon@n0255 ~]$ srun --mpi=pmi2 -p hpib -n 10 mpitest
Hello world: rank 0 of 10 running on n0317
Hello world: rank 1 of 10 running on n0317
Hello world: rank 2 of 10 running on n0317
Hello world: rank 3 of 10 running on n0317
Hello world: rank 4 of 10 running on n0317
Hello world: rank 5 of 10 running on n0317
Hello world: rank 6 of 10 running on n0317
Hello world: rank 7 of 10 running on n0317
Hello world: rank 8 of 10 running on n0317
Hello world: rank 9 of 10 running on n0317

使用sbatch提交脚本如下(mpitest.sbatch):
#!/bin/bash
#SBATCH -n 10
#SBATCH -p hpib
srun --mpi=pmi2 -p hpib -n 10 mpitest

提交:
sbatch mpitest.sbatch