Environments
One of the most powerful characteristic of CONDA is the use of virtual environments to set-up your project software needs. Each of them is independent, so the management of different versions of the same software can be done in a straight forward way.
By default there are some basic environments installed in the system separated by fields of knowledge:
| Environment name | Description |
|---|---|
machine-learning | Environment with some of the most important software of machine learning, such as TensowFlow, Keras, Pytorch or Sklearn. |
machine-learning-gpu | Machine learning environment with support for GPU usage. |
bio-computation | Environment with bio-computational packages like BioPython, BioPerl and ClustalW. |
quantum-chem | Environment with packages that allow chemical calculations like Pyscf . |
However if you need a new package or some specific version, a custom environment can be easily created by the user himself. Those custom environments are installed in ~/.conda/env/<env_name> by default.
Useful commands
| Command | Description |
|---|---|
conda env list | List available environments. |
conda activate env_name | Activate the environment env_name. |
conda list | List installed packages in the active conda environment. |
conda create -n my_env | Create an environment called my_env. The custom creation path is ~/.conda/envs/my_env. |
conda install package | Install the package in the active conda environment. |
SLURM Submit script example
This script example has been generated using the Job Script Generator.
#!/bin/bash
#SBATCH -J conda_example
#SBATCH -e conda_example.err
#SBATCH -o conda_example.out
#SBATCH -p std
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --mem=2000MB
module load apps/conda/3
INPUT_DIR=${SLURM_SUBMIT_DIR}
OUTPUT_DIR=${SLURM_SUBMIT_DIR}
cd $SCRATCH
cp -r $INPUT_DIR/* $SCRATCH
conda activate environment_name
python example.py
cp ./* $OUTPUT_DIR
Sbatch options:
- -J: Specify a name for the job allocation. The default is the name of the batch script.
- -e: Specify a name for the error output file.
- -o: Specify a name for the output file.
- -p: Specify the name of the partition (queue) where the job will be submitted. The default is std.
- --nodes: Number of nodes requested for allocation.
- --ntasks: Number of processes requested for allocation.
- --mem, --mem-per-cpu: Memory allocated per node/core respectively. If it is not specified SLURM associates:
- 3998MB per requested core in std and gpu nodes.
- 24180MB per requested core in mem nodes.