Help:Jupyter notebook

Jupyter notebook is a web-based interactive computational environment for creating Jupyter notebook documents.

Software availability

anywhere python is installed

Other related software

julia

Running Jupyter Notebook on a slurm cluster

There are two ways to run Jupyter Notebook on a slurm cluster:

• Run it on the login node, and have it submit slurm jobs to accomplish computation (this requires complex configuration and may be added here later)
• Run it inside a slurm job

Running Jupyter Notebook as a slurm job

This method does not require any special configuration of Jupyter to make it aware of slurm, and is much simpler to set up. However, since it is not aware of slurm, it is your responsibility to exit Jupyter Lab or kill the job when you are done using it so that others can use the gpus.

To run Jupyter Lab in CRCV cluster, follow these steps:

• run the following in a slurm job, modify this to suit your needs:

#!/bin/bash
#SBATCH -c4
#SBATCH --gres=gpu:1
#SBATCH -p gpu

## modify these modules for your needs
module load cuda anaconda3
# source activate <your env>
jupyter lab --ip=0.0.0.0

• Check your slurm log file for a line like this:

 or copy and paste one of these URLs:
       http://(c1-2 or 127.0.0.1):8888/?token=XXXXXXXX

• Create a tunnel between your local computer and the cluster using the highlighted items above. Note: the -L option wants local port blue), then remote hostname (green), and remote port (red) as shown in these examples. Note: you can pick the local port number -- change this if you have a conflict or want to open multiple tunnels.

You can do this by opening a new ssh session like this:
ssh -N -L 8080:c1-2:8888 crcv.eecs.ucf.edu

Or, add the tunnel to an existing ssh session. If you are using mobaXterm or putty, you need to do this with the GUI menus. If you are using a command line ssh, do it this way:

• Press return ~C return and you should get a ssh> prompt
• enter the -L option, as listed above:

-L 8080:c1-2:8888

• Open Jupyter Lab by constructing a URL from the pieces above and open it in your local browser. For example:

http://127.0.0.1:8080/?token=your token sequence found in the slurm log file 

Now you should be able to see Jupyter Lab with all your cluster directories on the environment you activated in slurm job and with allocated GPUs.

Don't forget to exit Jupyter Lab or kill the job when you are done using it so that others can use the gpus.