Topics Map > Services > Research Computing and Support > CCAST
Using Python on CCAST
Python installations available on CCAST
There are several ways to run Python on CCAST. Below is a list of the different Python installations available and how to use them.
Base (operating system) Python
Python is installed standard as part of the operating system on all CCAST servers (login and compute nodes). These versions are available simply by logging into the system with a terminal and running python
. This will launch a Python interpreter from one of the following locations:
/usr/bin/python
/usr/bin/python2
/usr/bin/python3
On newer systems (e.g. Thunder Prime) running python
will default to Python version 3, whereas on older systems (e.g. Thunder) running python
will default to Python version 2. When in doubt, the best practice is to explicitly run the version you want by calling either python2
or python3
.
The default, operating system Python installation is good for light scripting, or for code that only uses what’s available in the base Python distribution. If you need to make use of third-party packages in your Python code, consider using one of the other Python installations mentioned below.
Basic versioned Python modules
There are several Python versions available via the modules
framework. To view Python versions available via the modules system, run module avail python
. Here are the stable versions available as modules on CCAST:
python2/2.7.18
python/3.8.6-gcc-2pmf
python/3.8.6-intel-uly7
python/3.9.9
python/3.10.14
python/3.11.9
python/3.12.3
To load one of these modules, e.g. python2/2.7.18
, run module load python2/2.7.18
in your CCAST terminal session.
Like the base operating-system Python versions, these only include a basic Python distribution with minimal third-party packages. The benefit, however, is that these are strictly versioned and are consistent across nodes. Whereas, the base operating-system Python distributions are subject to version changes if the systems are updated.
Creating custom Python environments
There are two different ways CCAST users can create their own Python environments using venv
virtual environments. Instructions for each are provided below.
Python virtual environments
Python virtual environments use the pip
package manager and a basic versioned Python distribution. Here are the basic steps:
1. Load a Python3 module
module load python/3.12.2
2. Create the virtual environment
python -m venv myenv
This will create a directory named myenv
in your current working directory, containing a link to the Python interpreter as well as folders for libraries and other supporting files. For consistency, virtual environment folders should not be moved once created, so make sure you are in the directory you want before creating the environment.
If the above command succeeds, you can now unload the Python module:
module unload python/3.12.2
3. Activate your virtual environment
source myenv/bin/activate
Upon activating the virtual environment, you should see your terminal prompt change from something like this:
[user.name@login0003 ~]$
to something like this:
(myenv) [user.name@login0003 ~]$
This is to remind you that you are “inside” the virtual environment.
4. Upgrade pip and install packages
With a new virtual environment, it is always a good idea to first update the pip
package manager:
pip install --upgrade pip
After this, you can install the packages you need. For example, to install numpy
:
pip install numpy
5. Exiting the virtual environment
When you are done working in the virtual environment, simply run:
deactivate
Your terminal prompt will revert to normal to indicate that you are no longer in the virtual environment.
To use the environment again in the future, rerun the source
command from step 3.
Integrating custom Python environments with Jupyter Notebook
In addition to running Python jobs via the batch scheduler, CCAST users can run Python interactively through the Jupyter Notebook app in Open OnDemand. To launch a Jupyter session, login to CCAST’s Open OnDemand service and select “Jupyter” from the “Interactive Apps” menu.
By default, Jupyter launches with one of CCAST's provided modules, in order to provide a standard environment with popular packages for beginners to get started quickly. More advanced users may want to integrate their own custom Python environments with the Jupyter app. To do so, follow these steps:
Activate your Python environment
For virtual environments:
source path/to/virtual_environment/bin/activate
Install the environment as a ipykernel
For both virtual environments:
python -m ipykernel install --user --name=MyEnvName
Note: The name you assign with the --name
command will be the name that appears in the Jupyter app.
4. Launch a Jupyter Notebook session and create a new notebook with your kernel
In the Jupyter interface, select the “New” dropdown on the right and you should see your custom kernel. Selecting it will launch a new notebook session with your custom environment.
5. Managing Jupyter kernels
To see a list of configured kernels from the terminal:
jupyter kernelspec list
To remove a kernel:
jupyter kernelspec remove MyEnv