Devlog: Integrating ComfyUI with Open WebUI

What exists already

There are seemingly no helm charts available except for a few “homemade” ones, with little usage.

So we’ll make our own manifests. A google search for ComfyUI images gives us saladtechnologies/comfyui, which has a decent amount of downloads and updated images coming out at regular intervals. The images are stateless, meaning they come prebuilt with all model parameters. The large Stable Diffusion 3.5 image is about 22GB, and should fit perfectly inside my RTX 3090 card.

AI-generated manifests

Gemma2

apiVersion: apps/v1
kind: Deployment
metadata:
  name: comfyui-deployment
  namespace: openwebui
spec:
  replicas: 1
  selector:
    matchLabels:
      app: comfyui
  template:
    metadata:
      labels:
        app: comfyui
    spec:
      containers:
      - name: comfyui
        image: saladtechnologies/comfyui:comfy0.3.12-api1.8.1-sd3.5-large
        ports:
        - containerPort: 8188
---
apiVersion: v1
kind: Service
metadata:
  name: comfyui-service
  namespace: openwebui
spec:
  selector:
    app: comfyui
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8188
  type: LoadBalancer

Except for the service type this looks fine — let’s smack it into Kubernetes and see what happens.

It took about 20 minutes to pull the image, but when pulled the container ran flawlessly.

Configuring Open WebUI

Going to the admin settings under “Images” and setting the URL to the ComfyUI service. Copy-pasting in the workflow.json from saladtechnologies’ GitHub.

In the ComfyUI Workflow Nodes set the prompt node id to the same as in the workflow.json.

Generate a colorful cartoon sketch drawing of a hands-in-the-air happy developer that has gotten his code to work late at night in his home office. With black IT equipment and a custom build PC and multiple screens. Outside the window is a starry night sky with a full moon. The desk has a few coffee cups.

Success with an asterisk

It worked out with a lot less work than expected. A downside is that ComfyUI loads the full model into VRAM at startup, which doesn’t leave a lot left for the LLM. The model is kept in VRAM even when it is idle. I guess we have to scale down ComfyUI when we’re not using it. At least for now.

Finally, some probes

Let us delegate the task to qwen2.5-coder:32b:

# Me: Can you add a health and readiness check to this deployment manifest?
# The health probe is available at the /health route on the same port as the comfyui container
# The readiness probe is available at the /ready route on the same port as well

# Qwen:
livenessProbe:
  httpGet:
    path: /health
    port: 8188
  initialDelaySeconds: 30
  periodSeconds: 10
readinessProbe:
  httpGet:
    path: /ready
    port: 8188
  initialDelaySeconds: 5
  periodSeconds: 10
  successThreshold: 1
  failureThreshold: 3

The probes were actually on port 3000 — my bad. After correcting for the wrong port number and adjusting the failure threshold to fit a readiness time of about a minute, we are DONE!