Conda

We provide the miniconda package manager mostly as a legacy toolset for users that have existing conda environments. However for creating new environments we highly encourage you to try micromamba as a replacement. Micromamba is a C++ executable with a powerful command line interface, which is mostly a drop in replacement for conda. Except for some legacy options and corner cases.

It has muiltiple benefits that make it the recommended tool on terrabyte compared to conda. First of all it does not need a base environment and therefore does not come with a default Python stack, which might not be needed. Furthermore its C++ solver is optimzed and parallelized utilizing the multiple cores on terrabyte way better when installing or updating larger environments, also downloading and installing packages in parallel. It also does not use the anaconda default channel, which has licence restrictions on its use.

Loading miniconda via the modules system

On terrabyte, miniconda is available from the module system and newer versions are provided on the terrabyte dss share. There are multiple versions of miniconda available. To make it accessible in your shell, use following commands:

# consider adding the module use line to your ~/.bashrc to always make terrabyte modules available 
module use /dss/dsstbyfs01/pn56su/pn56su-dss-0020/usr/share/modules/files/
module load miniconda3

Create new conda environment

After loading miniconda, a new environment called exampleEnv could be created by running

conda create -n exampleEnv -c conda-forge <list of packages>
# alternative to install shared Env outside of your home eg´.g. in DSS Container
conda create -p </dss/.../exampleEnv>  -c conda-forge <list of packages>

If the new environment contains the package jupyterlab it can directly be used to start an interactive Notebook via the portal. In the same way any micromamba environment created via the portal can also be activated from the login node or from inside a SLURM Job.

Installing extra packages into existing environment

Additional packages can be installed or updated at a later date via

conda install -n exampleEnv -c conda-forge <list of additional packages>

Running commands from a micromamba environment

Contrary to the recommendation of micromamba and conda we do not recommend placing the shell eval script inside your ~/.bashrc or similar as the activation "vodoo" of mamba can adversly interact with the similar approaches of the module system leading to hard to debug and unexpected causalities depending on the order of module loading and micromamba environments. This is especially true when using the portal as here we have observed thses issues. Therefore our suggestion is to not use activate but rather directly run a command or start an new shell via:

conda run -n exampleEnv bash

#or
conda run -n exampleEnv python myscript.py

# or initalize the environment inside your SLURM script (least recommended)
eval "$(conda shell hook -s posix)"
conda activate exampleEnv
python myscript.py

# another alternative to initalize the environment inside your SLURM-script (least recommended)
source activate exampleEnv
python myscript.py