OpenMPI
If you submit your jobs with sbatch and your job loads an OpenMPI module, you can issue your mpirun commands without an option describing the resources to be used (such as -np, -H, -npersocket, etc.). OpenMPI has indeed been configured to retrieve this information directly from Slurm.
In order to optimize the performance of your jobs, it is therefore advisable to configure the options for booking resources with Slurm. If you do not give a sufficiently precise definition of the reserved resources, your tasks may be dispatched to different nodes (via the queue cpucourt and if the cluster is widely used) and therefore you will lose performance. It is therefore recommended to use options that specify the reservation, such as:
--nodes=1
--ntasks=4
--ntasks-per-node=4
--ntasks-per-socket=4
Here, we reserve 4 cores on the same node and the same processor. For information, the computation nodes (queues cpucourt and cpulong) are equipped with 2 processors (socket), each having 20 cores.
Here is an example of a job declaration with Slurm where the code will run on 2 nodes (with 10 cores used on each socket):
#!/bin/bash #SBATCH --job-name=check_mpi #SBATCH --output=check_mpi.txt #SBATCH --time=10:00 #SBATCH --account=project_name #SBATCH --ntasks=40 #SBATCH --ntasks-per-node=20 #SBATCH --nodes=2 #SBATCH --ntasks-per-socket=10 module purge module load openmpi/4.1.7 mpirun program.mpi
Intel MPI
Job example:
#!/bin/bash #SBATCH --job-name=test_mpi #SBATCH --output=mpi.txt #SBATCH --ntasks=4 #SBATCH --time=10:00 #SBATCH --account=projectname #SBATCH --constraint=intel module purge module load intel/2020-cluster-xe mpirun -bootstrap slurm -n $SLURM_NTASKS /home/username/jobs/my_code.mpi
Click here to read the official Intel MPI documentation.