If you need to install your own programs/modules in python you can do it via creating vms. Some programs may need specific versions of modules to work correctly and may cause incompatibilies with the current system library. By making Vms you can ensure an application can create a stable environment to run.

Loading Anaconda Python

Two versions of the anaconda distribution of python are loaded on Arc; The Anaconda module runs python 2.7, and Anaconda3 runs Python 3.6. You load them like any other module on shamu.

module load anaconda2

or

module load anaconda3

Creating a VM

After you load your preffered Anaconda module you can create a new python vm with the name 'myvm' by running the command:

conda create -n myvm python

Select 'y' and let it install to the default location. You can create multiple vms to suit your needs. You can list your Vms via this command:

conda env list

The asterisk indicates which one is active.

[abc123@login02 ~]$ conda env list

# conda environments:

#

base* /apps/anaconda3/5.2.0

myvm/work/abc123/.conda/envs/myvm

Activating a VM

To select or "activate" a vm, enter the following command:

conda activate myvm

This will change the prompt to let you know which Vm you're using:

[abc123@login02 ~]$ conda activate myvm

(myvm) [abc123@login02 ~]$ which python

~/.conda/envs/myvm/bin/python

You are still logged in to your regular session on Arc, and everything works as normal (submitting jobs, opening files, etc). The prompt indicates that your VM is active so the path to Python is now the one from your VE. You have complete ownership over the virtual environment, which means you can install applications whenever you'd like. See the conda and pip sections further down on the page for more details.

Exiting your VM

To exit the VM type:

conda deactivate

Deleting a VM

If you want to delete your VM, enter the following command:

conda env remove -n myvm

Where 'myvm' is the name of the VM you want to delete and then select 'y' at the prompt to confirm.

Installing Applications via conda

The good thing about virtual environments you create in your workspace is that you have ownership permissions over them. When you activate a VE, you can install programs without any administrator intervention. Both conda and pip have depdency checks, so performing and install in your VE will autofetch whatever dependencies the application needs and install them for you.

To install via conda, you just do:

conda install <package name>

You can also add conda repositories so that it will autofetch from more places by adding a channel to it via this command:

conda config --add channels <channel name>

Some popular channels for scientific computing resources are: conda-forge and bioconda.

Installing Applications via pip

pip pulls applications from the Python package index (https://pypi.org/). To install via pip you perform this command in your activated Python VE:

pip install <package name>

Installing from Python Source

Alternatively if all you can find is Python source code, you can still perform the install to your activated Python VE. Simply perform these commands after you change to the directory (cd) with the Python source:

python setup.py build

python setup.py install