Topics Map > Services > Research Computing and Support > CCAST

Running Containers on CCAST using Singularity

Overview and tutorial for using Singularity, a Linux containerization software, on CCAST: working with image files, installing on a local machine, and running scripts on CCAST.

    Singularity is an open-source computer program that allows for applications to be deployed on Linux as containers: lightweight, stand-alone, and executable packages for an application, containing everything that the application needs to run. Similar to Docker, Singularity allows for the creation of containers from a simple configuration file called a definition, or ready-made from an online repository. Singularity allows for the deployment of software containers on CCAST, allowing users to quickly deploy software locally and also to utilize features such as GPU hardware acceleration.

    While the full Singularity user documentation is available here, this document is a brief overview of Singularity and its use on CCAST.

    Running prebuilt containers on CCAST

    To use Singularity on CCAST, load the singularity module:

    module load singularity

    Then use the singularity command to interact with Singularity. For example, to see the available commands, use the singularity help command:

    singularity help

    To see the help for a specific command, use the singularity help <command> command:

    singularity help pull

    Singularity interacts with containers, which appear on the filesystem as .sif files. Users can pull an existing container from an online repository or build one themselves.

    Downloading a Pre-built Singularity Container

    Pulling (i.e. downloading) a Singularity container copies a ready-made container from an external, internet-based source. To pull a Singularity container, use the singularity pull command after loading the singularity module:

    singularity pull <container>

    Here, the can be either a container name or a container name with a tag. For example, to pull the container ubuntu:latest from the Official Docker Hub Source, use the command:

    singularity pull docker://ubuntu:latest

    This creates a local, ready-to-run containerized version of the Ubuntu distribution, named ubuntu_latest.sif by pulling it from Docker Hub.

    Finding Containers

    Many online repositories of containers exist, including:

    Running a Singularity Container

    To run a Singularity container, use the singularity exec command:

    singularity exec <container> <command>

    This runs the command <command> inside the container <container>. For example, using the previously-created container:

    singularity exec ubuntu_latest.sif cat /etc/lsb-release

    This runs cat /etc/lsb-release inside the container ubuntu_latest.sif, which prints the version of Ubuntu inside the container.

    Running an Interactive Shell

    To run an interactive shell inside a container, use the singularity shell command:

    singularity shell <container>

    This runs an interactive shell inside the container <container>.

    Example: PBS scripting Singularity

    Singularity containers can be run from within PBS scripts, and thus leverage the power of GPU acceleration for HPC jobs. The entire workflow of setting up and running can be performed using the CCAST scheduler. For example, the following PBS script runs a containerized version of the gromacs molecular dynamics package to perform a simulation of human alcohol dehydrogenase solvated in water, using 4 GPUs on CCAST:

    #PBS -q gpus
    #PBS -N gromacs-test
    #PBS -l select=1:mem=16GB:ngpus=1:ncpus=8:mpiprocs=4:ompthreads=2
    #PBS -l walltime=00:30:00
    #PBS -j oe
    #PBS -W group_list=<YOUR GROUP HERE>
    ## Load the singularity module
    module load singularity
    module load wget
    ## Change to the scratch directory and create a working directory
    cd $SCRATCH
    mkdir gromacs-test
    cd gromacs-test
    ## Copy the input files to the working directory
    tar -xzf ./GROMACS_heterogeneous_parallelization_benchmark_info_and_systems_JCP.tar.gz GROMACS_heterogeneous_parallelization_benchmark_info_and_systems_JCP/adh_dodec/ --strip-components=2
    ## pull the container
    singularity pull docker://
    ## Run the container
    APP="singularity run --nv -B ${PWD}:/host_pwd --pwd /host_pwd ${SIF}"
    ${APP} gmx grompp -f pme_verlet.mdp
    ${APP} gmx mdrun -v -nsteps 100000 -resetstep 90000 -noconfout -ntmpi 4 -ntomp 2 -nb gpu -bonded gpu -pme gpu -npme 1 -nstlist 400 -s topol.tpr
    ## Copy the output files back to the home directory
    cd ..
    cp -r gromacs-test $HOME
    ## Clean up
    rm -rf gromacs-test
    exit 0

    Installing Singularity on your local machine

    To create custom singularity containers for deployment onto CCAST, you will need to install Singularity on your local machine. While Singularity is Linux software, it can be installed on Mac OSX and Windows using a virtual machine. To install Singularity on your local machine, follow the instructions here. In brief, you can get Singularity working on your machine with the following:


    Singularity is easiest installed via the command line with the following commands, depending on your distribution:

    # Red Hat, Rocky Linux, CentOS, and Fedora
    $ sudo yum install -y epel-release
    $ sudo yum update
    $ sudo yum install singularity-ce
    # Ubuntu, Debian, Mint, and other Debian-based distributions
    $ sudo apt install -y software-properties-common build-essential \
        uuid-dev libgpgme-dev squashfs-tools libseccomp-dev wget pkg-config git cryptsetup-bin golang
    $ wget | sudo dpkg -i -
    # Arch, Manjaro, Artix, and other Arch-based distributions
    # using yay
    $ yay -S singularity-ce
    # using pkgbuild
    $ git clone
    $ cd singularity-ce && makepkg -si


    While Singularity is not available for Windows, it can be installed in a Linux virtual machine. Two options are available:

    Vagrant and VirtualBox

    Vagrant is a tool for managing virtual machines, and VirtualBox is a virtual machine manager. Vagrant and Virtualbox can be easily installed on Windows using the Windows Package Manager (winget) using powershell:

    > winget install virtualbox
    > winget install vagrant

    Then, to install a Linux virtual machine, run the following commands:

    > vagrant.exe init ubuntu/latest
    > vagrant.exe up

    The above would install the latest version of Ubuntu. To access the virtual machine, run the following command:

    > vagrant.exe ssh default

    You can then install Singularity on the virtual machine using the Linux instructions above.

    Windows Subsystem for Linux (WSL)

    Windows Subsystem for Linux (WSL) is a compatibility layer for running Linux natively on Windows. To install WSL on Windows, open a Powershell terminal and run the following commands:

    > dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
    > wsl --install

    This first enables WSL, then installs the software. Then, to install a Linux virtual machine, run the following commands:

    > wsl --list --online
    > wsl --install -d Ubuntu

    The above would install the latest version of Ubuntu. To access the virtual machine, run the following command:

    > wsl -d Ubuntu

    You can then install Singularity on the virtual machine using the Linux instructions above.

    Mac OSX

    Similarly to Windows, Singularity is not available for Mac OSX, it can be installed in a Linux virtual machine. The easiest method is to use homebrew to install VirtualBox and Vagrant using the following commands in the terminal:

    /bin/bash -c "$(curl -fsSL"
    brew install --cask virtualbox vagrant

    Then, Vagrant can be used to install a Linux virtual machine, and Singularity can be installed on the virtual machine using the Linux instructions above.

    Example: Creating Singularity Containers

    Singularity containers can be created from a definition file, which is a simple text file containing instructions for building the container, including the installation of software. For example, the following definition file creates a container with the latest version of Ubuntu, then installs the neovim text editor:

    Bootstrap: docker
    From: ubuntu:latest
        apt update
        apt install -y neovim
        export EDITOR=nvim
        export LC_ALL=C
        exec nvim "$@"

    To build this container, save the above text to a file named neovim.def, then use the singularity build command:

    singularity build neovim.sif neovim.def

    This creates a container named neovim.sif from the definition file neovim.def, and creates a default action which can be run for the container. To run the container, use the singularity run command:

    singularity run neovim.sif

    Works Referenced for Gromacs Example

    • Berendsen, H.J.C., van der Spoel, D. and van Drunen, R., GROMACS: A message-passing parallel molecular dynamics implementation, Comp. Phys. Comm. 91 (1995), 43-56.
    • Lindahl, E., Hess, B. and van der Spoel, D., GROMACS 3.0: A package for molecular simulation and trajectory analysis, J. Mol. Mod. 7 (2001) 306-317.

    Keywordsacceleration ccast containerization cuda docker gpu gpus gromacs hpc how-to installation instructional linux parallelism prime scientific singularity tutorial   Doc ID128518
    OwnerStephen S.GroupIT Knowledge Base
    Created2023-05-24 11:39 CDTUpdated2023-08-09 15:18 CDT
    SitesIT Knowledge Base
    Feedback  0   0