Access the JupyterHub Slurm Server
We have a JupyterHub server running on top of Slurm that is very useful for spawning Jupyter Notebooks.
Login to the Hub
You can login to the JupyterHub interface here:
https://jupyterhub.prism
Your web browser may complain about a name mismatch on the SSL certificate but you can click past it, the connection is still secure.
There you will see a login page:
Login using your regular UNIX username and password.
Configure Your Notebook
Once you login, you can configure your notebook for its required resources in Slurm, and then launch your notebook, which will appear on a node Slurm chooses. If a node is free right then, your notebook will launch immediately. If the Slurm queue is full, you may have to wait for resources to become available.
You will see this page after logging in, which is the "Simple" setup page:
Here you can define more specific job parameters.
You will want to mostly accept the defaults if running a simple notebook. But if your notebook requires more resources, then kick up the max time or CPU/GPU count as appropriate. Remember Slurm will kill the notebook if it persists past the Job Duration parameter. Simply click the Start button on the bottom to launch the notebook.
If you need a more specific environment, click on the Advanced tab, which should yield this screen:
This allows for a more fine grained resource definition when requesting resources for your notebook. It also allows for a custom Jupyter environment to be defined, if you need one. The Default environment is fairly basic, so it may make sense to create your own custom environment. If you would like to do that, refer to Create a Jupyter Virtual Environment.
The Live Notebook
Once you start your notebook (give it some time, it can take 10-20 seconds to start based on your environment and resource requests), you should see your notebook appear as something like this in your web browser:
If the slurm queue is full, you may be waiting a while.
This is a notebook actually running on one of the cluster nodes, but the connection is proxied through the head node, and you are seeing it in your web browser. You can actually see your job running in slurm by looking at the output of the squeue command, if you are curious.
Terminate Your Notebook When Finished
When you are done, don't forget to terminate your notebook to free up the Slurm resources it was using! To close your notebook, click on File -> Hub Control Panel. Then from there click the red Stop My Server button. Your notebook will terminate and the associated Slurm job will also exit.
If you simply click "Logout", or "File -> Logout", it will log you out, but your notebook is still running! So next time you log in, it will take you right back to that notebook. If you are truly done, then kill your notebook via Stop My Server as described above.
If slurm sees the job is still running when the job's wall time is exceeded, slurm will kill your notebook automatically.
