Create your Python environment

We strongly encourage users to create their own Python environment with the Miniconda distribution rather than using the centralized Python distributions available with the cluster modules. You can work with the latest updates faster, get performance improvements and avoid possible incompatibilities between different modules.

With Miniconda, you can create your own Python environment and install the packages you need there. You can even repeat the operation for different projects and thus have an environment specific to each.

Miniconda is a minimalist version of Anaconda and includes a basic Python and the Conda package manager. You can then install additional packages by following the procedure below.

Prerequisites

To use the miniconda commands, you must first load the miniconda module on the login node:

module load miniconda

Create the environment

Create the environment by giving it a name (here my_env) and choosing its Python version (here 3.7):

conda create --name my_env python=3.7  

The environments you create are added to the /home/$USER/.conda/envs folder.

You can also install it in another directory. For instance, you can share an environment with the other users of your project. To do so, you should install it in your shared directory in /workspace/your_group_name:

 conda create python=3.7 --prefix /workspace/your_group_name

Activate your environment

# if you gave a name to your environment:
conda activate my_env
# if you created it with the prefix option:
conda activate /workspace/your_group_name

Add a package

conda install package_name

List your environments

conda env list

The asterisk in front of one of the listed environments indicates which one is currently in use.

Deactivate your environment

conda deactivate

Delete an environment

Here, we delete an environment called my_env:

conda env remove --name my_env

Job example

#!/bin/sh
#SBATCH --job-name=python_job_conda
#SBATCH --time=0-00:03:00
#SBATCH --account=project_name
#SBATCH --output=my_job_output.txt

module purge
module load miniconda

conda activate /home/user1/.conda/envs/my_env
python /home/user1/jobs/my_code.py
conda deactivate