Devlog: Setting up Open WebUI in K8s with GPU

Setting up a new node

Currently my juciest GPU is in my gaming PC. As I do not want to take it out, I have to bring K8s to my gaming Laptop.

Setting up dual boot with Ubuntu server on a 200gb partition that serendipitously was unpartitioned on a HDD.

Setting up NFS

Since my single node cluster uses node storage I need to do something to distribute it, so that I do not have to be bound to a node. Looking at several technologies, rook and longhorn looked promising. NFS won in the end for being the simplest to implement (probably).

First setting up an NFS server on my k8s server following ubuntu’s docs:

sudo apt install nfs-kernel-server
sudo systemctl start nfs-kernel-server.service

sudo echo "/data 192.168.1.203(rw,no_root_squash,no_subtree_check,sync)" >> /etc/exports

sudo exportfs -a

Setting up the client on the gaming PC node:

sudo apt install nfs-common

sudo mkdir /data

sudo echo "192.168.1.200:/data /data nfs rsize=8192,wsize=8192,timeo=14,intr" >> /etc/fstab

Problem: entries in fstab is mounted before the network controller is up. Another problem: /etc/if-up-d is no longer used in Ubuntu 24.

Solution: use networkd-dispatcher. Create the following script in /etc/networkd-dispatcher/routable.d/fstab:

#!/bin/bash
mount -a

Make the script executable and enable networkd-dispatcher:

sudo chmod +x /etc/networkd-dispatcher/routable.d/fstab
systemctl enable networkd-dispatcher.service

Installing containerd

…work in progress — original notes ended here. This post was migrated from the Jekyll site as part of the move to Astro.