If you want to run JupyterLab Notebooks within Slurm job, you can use one of these methods:
1- Open your Jupyter Notebook with JupyterLab.
2- From the “File” menu from the JupyterLab interface choose “Save and Export Notebook As” and select “Executable Script”. This step will save the Python script on your local computer.
3- Copy your exported code into HCC Swan cluster.
4- Now, you can create a SLURM submit file and load the system-wide module that corresponds to the system-wide kernel in the submit script.
5- Finally, you can run your Python script by adding python test.py in your SLURM submit script where test.py is the name of your Python file.
If you have used your custom kernel, then you will need to load the anaconda module using the command module load anaconda in the SLURM submit script and activate your custom conda environment as shown in https://hcc.unl.edu/docs/applications/user_software/using_anaconda_package_manager/#creating-custom-anaconda-environments.
In this method, ipython and nbconvert are required. The majority of system-wide kernels already have ipython and nbconvert installed. If that is not the case, please email us at hcc-support@unl.edu.
1- Create a SLURM submit file and load the system-wide module that corresponds to the system-wide kernel in the submit script.
2- Now, you can run your JupyterLab Notebook by adding ipython -c “%run test.ipynb” in your SLURM submit script where test.ipynb is the name of your Notebook.
If you want to use your custom kernel, please make sure ipython and nbconvert are installed in the conda environment. Then you will need to load the anaconda module using the command module load anaconda in the SLURM submit script and activate your custom conda environment as shown in https://hcc.unl.edu/docs/applications/user_software/using_anaconda_package_manager/#creating-custom-anaconda-environments.
In this method nbconvert is required. The majority of system-wide kernels already have nbconvert installed. If that is not the case, please email us at hcc-support@unl.edu.
1- Create a SLURM submit file and load the system-wide module that corresponds to the system-wide kernel in the submit script.
2- Now, you can run your JupyterLab Notebook by adding jupyter-nbconvert –execute –clear-output test.ipynb in your SLURM submit script where test.ipynb is the name of your Notebook.
If you want to use your custom kernel, please make sure nbconvert is installed in the conda environment. Then you will need to load the anaconda module using the command module load anaconda in the SLURM submit script and activate your custom conda environment as shown in https://hcc.unl.edu/docs/applications/user_software/using_anaconda_package_manager/#creating-custom-anaconda-environments.