...
Jupyter is an open-source tool that provides an interactive computing environment, enabling users to create and share documents containing live code, equations, visualizations, and narrative text. Jupyter notebooks are especially popular in data science, machine learning, and scientific computing because they combine code and results in a single, user-friendly interface.
...
Quick Jobs with Jupyter
...
...
This method does not support custom conda environments. Please continue reading to follow the instructions in Jupyter through Interactive Session and so on to access Jupyter with your custom environments.
You can connect to the HPC Jupyter Domain here:
https://jupyter.storrs.hpc.uconn.edu
Log in with your NetIDand NetID and password:
...
Select from the following quick options for partition, number of cores, and time limit and memory allocation:
...
| Warning |
|---|
If login fails, please email hpc@uconn.edu to have an administrator recreate your user certificate. |
JupyterHub through Interactive Session
In the terminal on the cluster, use srun to request resources through SLURM for an interactive session. Here we ask for 1 node, 5 cores, 5G of memory, in the general partition, nodes with cpu only and 2 hours of runtime:
| Code Block |
|---|
srun -N 1 -n 5 --mem=5G -p general --constraint=cpuonly --time=00:02:00:00 --pty bash |
| Code Block |
|---|
srun: job 6186040 queued and waiting for resources
srun: job 6186040 has been allocated resources
Agent pid 513661
Your current list of jobs:
JobID Partition QOS JobName User Submit Elapsed State NNodes NCPUS NodeList Start
------------ ---------- ---------- ---------- --------- ------------------- ---------- ---------- -------- ---------- --------------- -------------------
6186040 general general bash jht12345 2024-11-12T13:18:28 00:00:06 RUNNING 1 5 cn457 2024-11-12T13:18:29
|
Next, load the “jupyterhub” module, you can check available modules using the command module avail and load them with module load
| Code Block |
|---|
module avail jupyterhub
------------------------------------------- /cm/shared/modulefiles -------------------------------------------
jupyterhub/live jupyterhub/test4 |
| Code Block |
|---|
module load jupyterhub/live
Loading jupyterhub/live
Loading requirement: pre-module post-module gcc/13.2.0 |
| Note |
|---|
If you run in any module conflicts, use |
Navigate to your desired working directory and launch jupyter
| Code Block |
|---|
jupyter-lab --no-browser --ip='*' --port=8890 |
You will be greeted with a very large output. The main thing to look for is the connection is successful and that a server is running on the port chosen. At the bottom of the ouput, you will have links to access your jupyter page but one more step is needed to allow it to load.
| Note |
|---|
If you run into an error or the output is blank after attempting to launch jupyter-lab, it is likely that the port you selected is occupied. In this case, try a higher port number such as 8891, 8892 or more. Generally, ports between 8888 to 9000 work well. |
Set up SSH Port Forwarding
In a separate terminal window to access the jupyter server from your local machine. Use the following command with the correct hostname and selected port filled in.
| Code Block |
|---|
ssh -L <port>:<nodename>:<port> <your_netid>@hpc2.storrs.hpc.uconn.edu |
From our example above, we landed on cn457 and we chose 8890 as our port:
| Code Block |
|---|
ssh -L 8890:cn457:8890 jht123456@hpc2.storrs.hpc.uconn.edu |
Now to finally access the server, you can either of the 2 links provided in the output of your original terminal.
...
Custom Conda Environments in Jupyter
For information on how to create custom conda enviroments, please visit our guide:
Miniconda Environment Set Up - Storrs HPC - UConn Knowledge Base
Begin by activating/creating your desired environment:
| Code Block |
|---|
conda create -n myenv |
| Code Block |
|---|
conda activate myenv |
Once you are in the enirvonment, install ipykernel with the following command:
| Code Block |
|---|
conda install ipykernel |
Add the environment to your jupyter kernels:
| Code Block |
|---|
python -m ipykernel install --user --name=myenv --display-name "myenv" |
Follow the previous instructions to launch jupyter. You should now be able to select the environment we just added:
...
The following advanced options are also available:
| Info |
|---|
To access the priority or priority-gpu partitions, make sure to include the proper “account” and “qos” in “Extra options”. |
...
Accessing Shared Storage
Due to limitations from jupyter the file browser and initial working directory are limited to the users home directory.
There are several methods to access shared storage.
Within a Notebook:
| Code Block |
|---|
import os
os.chdir('/shared')
os.chdir('/scratch')
or
%cd /shared
%cd /scratch |
In the File Browser:
Create a symlink in the users home directory.
| Code Block |
|---|
ln -s /path/to/directory <directoryName>
e.g.
ln -s /shared/ ~/shared |
Custom Environments
If you require a non-default python environment we provide setup examples here: custom environments.