Micromamba
micromamba is a tiny version of the mamba package manager. It is a C++ executable with a powerful command line interface , which is mostly a drop in replacement for conda. However it has muiltiple benefits that make it the recommended tool on terrabyte compared to conda. First of all it does not need a base environment and therefore does not come with a default version of Python. Furthermore its c++ solver is optimzed and parallelized utilizing the multiple cores on terrabyte way better when installing or updating larger environments.
loading micromamba via modules system
On terabyte micromamba is available from the module system and newer versions are provided in the terrabyte dss share. There are multiple versions of micromamba available and
To make it accessible in your shell run:
# consider adding the module use line to your ~/.bashrc to always make terrabyte modules available
module use /dss/dsstbyfs01/pn56su/pn56su-dss-0020/usr/share/modules/files/
module load micromamba
create new micromamba environment
After loading micromamba a new environent called exampleEnv could be created by running
micromamba create -n exampleEnv -c conda-forge gdal python ... jupyterlab
# alternative to install shared Env outside of your home eg´.g. in DSS Container
micromamba create -p </dss/.../exampleEnv> -c conda-forge gdal gdal python ... jupyterlab
If the new environment contains the package jupyterlab it can directly be used to start an interactive Notebook via the portal. IN the same way any micromamba environment created via the portal can also be activated from the login node or from inside s SLURM Job.
running commands from a micromamba environment
Contrary to the recommendation of micromamba and conda we do not recommend placing the shell eval script inside your ~/.bashrc or similar as the activation "vodoo" of mamba can adversly interact with the similar approaches of the module system leading to hard to debug and unexpected causalities depending on the order of module loading and micromamba environments. This is especially true when using the portal as here we have observed thses issues. Therefore our suggestion is to not use activate but rather directly run a command or start an new shell via:
micromamba run -n exampleEnv bash
#or
micromamba run -n exampleEnv python myscript.py <script params>
# or initalize inside your script (least recommended)
eval "$(micromamba shell hook -s posix)"
micromamba activate exampleEnv
python myscript.py <script params>
For further examples and details on all micriomamba subcommands and options please consult the micromamba documentation.