Accessing VSC & Setting Up for LLM Workflows

This tutorial will guide you step-by-step through setting up a working environment on the VSC (Vlaams Supercomputer Centrum).
By the end, you’ll be ready to run your first SLURM job—such as downloading or serving a large language model (LLM) like Ollama—for your research.

This tutorial guides you through setting up your working environment on the VSC, using Visual Studio Code for a seamless experience.
You’ll learn how to connect remotely, organize your workspace, configure Python, and download models using SLURM.

1. Connect to VSC via VS Code

Why?
Using VS Code’s Remote - SSH lets you work on VSC almost as if it’s your local computer i.e., you can edit scripts, submit jobs, and manage files.

1.1 Prerequisites

  • Visual Studio Code (latest version)
  • Remote - SSH extension (ms-vscode-remote.remote-ssh)

1.2 SSH Key Access

You need to be able to log into VSC via SSH.
If you haven’t set this up, do the following in your local terminal:

ssh-keygen -t ed25519 -C "your_email@university.be"
ssh-copy-id vscXXXX@login.hpc.kuleuven.be
  • Replace vscXXXX with your VSC username.
  • Authorize your key on the VSC firewall.
  • Test SSH:
ssh vscXXXX@login.hpc.kuleuven.be

1.3 Connect in VS Code

  • Open VS Code
  • Press F1 (or Ctrl+Shift+P)
  • Type Remote-SSH: Connect to Host…
  • Enter:
ssh vscXXXX@login.hpc.kuleuven.be
  • Open a New Terminal in VS Code (it will run commands directly on VSC)

2. Organize your HPC Project

In your VS Code terminal (connected to VSC), run:

cd /data/leuven/377/vscXXXX/
mkdir -p llm_project/{code,data,models,logs}

This will create directories for:

  • code/ — Python and SLURM scripts
  • data/ — Input data
  • models/ — Downloaded models
  • logs/ — SLURM job logs and outputs

3. Set Up Python and Conda

3.1 Installing Miniconda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrc
conda --version

3.2 Load the Python Module

module load Python/3.10.4-GCCcore-11.3.0

3.3 Create and Activate Your Environment

conda create -n ollama_venv python=3.10 -y
conda activate ollama_venv

3.4 Install all neccesary packages

You can either use conda or pip for installation purposes.

conda install "package", ...
pip install python-dotenv

4. Downloading an Open Source LLM with SLURM

Note:
Never download large models interactively, Always use SLURM jobs to manage the download process on VSC.

4.1 Steps

  1. Navigate to your project directory:
    • cd /data/leuven/377/vscXXXX/
  2. Check that your SLURM script (ollama_setup.slurm) is present:
    • ls -l ollama_setup.slurm
  3. Submit the SLURM job to download the model:
    • sbatch ollama_setup.slurm
    • This command queues your job for execution on the cluster.
  4. Monitor your job’s status:
    • squeue -u $USER
  5. Watch live logs to track the download and check for errors:
    • tail -f ollama_download.out

4.2 Template

Below is a sample SLURM script for downloading an Ollama model.

#!/bin/bash -l        
#SBATCH --cluster=wice
#SBATCH --partition=batch_sapphirerapids
#SBATCH --time=02:00:00
#SBATCH --mem=128G
#SBATCH --account=xxxxx
#SBATCH --output=ollama_download.out

# the directory where models will be stored
export OLLAMA_MODELS=/staging/leuven/stg_00191/ollama_models

# the host and port for the Ollama server
export OLLAMA_HOST=127.0.0.1:xxxxx

# the Ollama server in the background
/data/leuven/377/vscXXXX/ollama/bin/ollama serve &

# wait a short time to ensure the server starts properly
sleep 15

# download the desired model from Ollama's repository
/data/leuven/377/vscXXXX/ollama/bin/ollama pull deepseek-r1:70b

# wait extra time to ensure the download completes before the job ends
sleep 600

What Each Section of the script does:

  • The #SBATCH lines set SLURM job options:

    • cluster: Specifies which HPC cluster to use.
    • partition: Chooses the spscific harware or queue/resource pool to be used.
    • time: Sets the maximum run time allowed for the job (HH:MM:SS).
    • memory: Allocates the required amount of RAM (e.g., 128G for 128 gigabytes).
    • account: Defines the billing/project group charged for resources.
    • output log: Specifies the file to write job output and errors for monitoring and debugging.

  • The export lines configure environment variables so Ollama knows where to store models and how to serve them.

  • The command ’‘/data/leuven/377/vscXXXX/ollama/bin/ollama serve &’’ starts the Ollama server in the background.

  • sleep 15 ensures the server has time to start before the model download begins.

  • ’‘/data/leuven/377/vscXXXX/ollama/bin/ollama pull deepseek-r1:70b’’ downloads the specified model i.e., deepseek-r1:70b

  • The final sleep 600 keeps the job running long enough for the download to complete, preventing early termination.

Note:
Once this process is complete, your model will be fully downloaded and ready for use in your analyses or experiments. If you wish to download multiple models, simply add additional download commands to the same SLURM script, they will be processed in order during the job.