Singularity is a container technology developed specifically for use on multi-tenant systems. A container allows you to run your software (or scientific workflow) in separation from the server’s environment. You can use Singularity in a very similar way you would use Docker.
There are several ways to build a container 1. This can be done from Singularity Hub:
singularity build lolcow.simg shub://GodloveD/lolcow
as well as from Docker Hub:
singularity build lolcow.simg docker://godlovedc/lolcow
or from a recipe file (typically named Singularity) which you prepared or generated yourself:
sudo singularity build lolcow.simg Singularity
The last one requires superuser privileges, so you can either build the image yourself on your personal computer and copy it to the server or ask an admin to build an image for you. Singularity runs natively on linux, but can be installed via Vagrant on other systems 2.
We might keep some popular container images in
/opt/software on Calcus.
The containers are executable, so you can simply do:
Or you can launch them through singularity:
singularity run mycontainer.img [arguments]
Binding 3 (mounting) is important if you want your container to interact with your files.
/dev are mounted by default.
There are two good ways to bind some folders. You can set the
SINGULARITY_BINDPATH environment variable. This seems to be the most convenient method:
Alternatively, you can use the
--bind option with
singularity run --bind /path/to/my/data:/data:ro,/path/to/my/output:/out ./some.img [arguments & options]
In both cases, you should specify the bind paths as a comma-delimited string using the format
src[:dest[:opts]]. Possible options are
ro (read only - used above for source data folder) and
rw (read/write, default, used above for output directory).
The overlay 4 feature has been enabled on our server, and as a result the destination bind points do not need to exist within the container (they will be created on an as-needed basis in an overlay file system without modifying the existing container 3). The use of /data and /out paths in the examples above follows the convention of BIDS apps.
Use case: BIDS Apps¶
You can use singularity to run BIDS Apps.
Example - MRIQC¶
First, prepare your image (versions change frequently, so it’s best if you do this yourself):
singularity build mriqc.simg docker://poldracklab/mriqc:latest
Then you specify bindings for your data (read only) and output folders:
and run QC like this:
./mriqc.simg [options] /data /out participant
For the list of options see 5
Example - FMRIPREP¶
Tested using singularity v2.5.2 and fmriprep v1.2.1.
Caveat: ANTS & number of threads¶
It appears that environment variables defined outside the container are visible inside it, and the
ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS (defines the number of threads used by ANTS, currently set to 8 on Calcus) seems to take precedence over the
--omp-nthreads options of fmriprep. It is therefore recommended that you unset it before running fmriprep (and probably also mriqc):
Caveat: freesurfer license¶
You need a freesurfer license to run fmriprep (even with
--fs-no-reconall). It has to be mounted inside the container and specified with
$HOME is mounted by default, you can put the license file in your home folder for simplicity.
Taking the above into consideration, I used the following script:
./fmriprep.img --participant_label sub-12 \
--nthreads 4 \
--omp-nthreads 2 \
--mem_mb 16000 \
--fs-license-file license.txt \
-w /work \
/data /out participant
See fmriprep docs 6 for the description of options.
Use case: Neurodocker¶
Neurodocker 7 is a wonderful tool which can generate Docker and Singularity recipes for most common neuroimaging applications. You can run it using docker or install it using pip.
Let’s assume that you want to work with ANTs version 2.3.1. You can create the recipe using docker:
docker run --rm kaczmarj/neurodocker:0.4.3 generate singularity \
--base debian:stretch --pkg-manager apt --ants version=2.3.1 \
or, if you installed through pip:
neurodocker generate singularity \
--base=debian:stretch --pkg-manager=apt --ants version=2.3.1 \
In both cases we redirected the output to a file named Singularity, so we can build:
sudo singularity build ants231.simg Singularity
Note that both docker itself and building from recipe in singularity require administrator privileges, so either execute these steps on your computer and copy the results or ask an admin to run them directly on Calcus.
You can combine different packages in one container. See the description and examples on Neurodocker’s github to see what you can do with it. And remember that if something is missing, you can use the generated recipes (they are just text files, try looking inside!) as a base and add your own stuff, e.g. by adding some
apt install lines.
Using containers created with Neurodocker¶
Let’s continue with the previous example. Executing the image:
will put you inside the container (technically speaking, it will start bash inside the container). You will have access to the programs installed within, instead of those installed natively on Calcus. You can test that:
Finally, you can finish working in the container and go back to the outer environment:
When inside the container, you will not be able to access your files unless mounted (explicitly or by default, see above for details).
You can either mount to a location existing within the container (likely such as
/mnt) or take use of the overlay feature and mount wherever you please. Assuming you want to work with data kept in
/opt/ssd/myfolder and mount it under the same path you can do:
Getting a hang of inside vs outside¶
Once again, let’s continue with the previous example. Assuming you are outside the container you can try the following:
# see where system installed programs are
# make sure you are not binding anything beyond default
# enter the container
# you have different ANTS at your disposal
# container has just the basics, no tmux inside
# home folder is mounted by default
# but not e.g. /opt/ssd
# go back outside
Build a container - Singularity 2.6 docs
Installation - Singularity 2.6 docs
Bind Paths and Mounts - Singularity 2.6 docs