Deploying DolphinScheduler 3.2.2 on Kubernetes with Rancher: A Step-by-Step Production Guide

[Chen Debra]

TL;DR​

This guide walks you through spinning up a production-ready Apache DolphinScheduler 3.2.2 cluster on Kubernetes + Rancher in under 30 minutes. We focus on three pain points every ops team meets in mainland China:

1. Image acceleration (no more DockerHub timeouts)
2. Dependency localization (charts & binaries cached inside the firewall)
3. Persistent storage (NFS-backed PVCs that survive pod restarts)

1. Prerequisites & Planning​

Component	Version	Purpose
Kubernetes	1.25+	Orchestration
Rancher	2.7+	GUI & helm chart repository
Helm	3.12+	Package manager
NFS Server	any	Shared storage for artifacts

Sequence Number	IP	HostName	Operating System	K8s Role	Description
1	192.168.255.140	harbor	Anolis OS 8.9 x86 64-bit	Harbor Service	Kubernetes Management Platform
1	192.168.255.141	rancher	Anolis OS 8.9 x86 64-bit	Rancher Service	Kubernetes Management Platform
2	192.168.255.142	master	Anolis OS 8.9 x86 64-bit	Control Node	Kubernetes Master
3	192.168.255.143	worker01	Anolis OS 8.9 x86 64-bit	Worker Node	Kubernetes Master High Availability Node, if high availability is not needed, this server can be not deployed
4	192.168.255.144	worker02	Anolis OS 8.9 x86 64-bit	Worker Node	Kubernetes Worker
5	192.168.255.145	worker03	Anolis OS 8.9 x86 64-bit	Worker Node	Kubernetes Worker
6	192.168.255.146	worker04	Anolis OS 8.9 x86 64-bit	Worker Node	Kubernetes Worker
7	192.168.255.147	worker05	Anolis OS 8.9 x86 64-bit	Worker Node	Kubernetes Worker

Environment Overview

2. Download & Patch the Helm Chart​

All commands are run from the Rancher local cluster shell.

# 1. Fetch the source
mkdir -p /opt/dolphinscheduler && cd /opt/dolphinscheduler
curl -Lo apache-dolphinscheduler-3.2.2-src.tar.gz \
  https://dlcdn.apache.org/dolphinscheduler/3.2.2/apache-dolphinscheduler-3.2.2-src.tar.gz
tar -xzf apache-dolphinscheduler-3.2.2-src.tar.gz

# 2. Enter the chart dir
cd apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler

2.1 Mirror Bitnami Charts (inside GFW)​

Edit Chart.yaml and replace GitHub raw URLs with fast mirrors:

-repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
+repository: https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami

Then refresh dependencies:

helm repo add bitnami-full-index \
  https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami
helm dependency update .
tar -xzf charts/*.tgz   # unzip postgresql, minio, zookeeper

3. Localize Container Images​

Create or edit values.yaml (only deltas shown):

# Global registry mirror for China
image:
  registry: registry.cn-hangzhou.aliyuncs.com/docker_image-ljx
  tag: 3.2.2

initImage:
  busybox: registry.cn-hangzhou.aliyuncs.com/docker_image-ljx/busybox:1.30.1

# Point each sub-chart to the same mirror
postgresql:
  image:
    registry: docker.io            # still needed for bitnami/postgresql
    repository: bitnami/postgresql
    tag: 11.11.0-debian-10-r71

minio:
  image:
    registry: docker.io
    repository: bitnami/minio
    tag: 2022.10.29-debian-11-r0

zookeeper:
  image:
    registry: docker.io
    repository: bitnami/zookeeper
    tag: 3.6.2-debian-10-r185

4. Configure NFS-backed Storage Class​

Save as sc.yaml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
  archiveOnDelete: "true"
---
# ... (full RBAC & Deployment omitted for brevity)

Apply it:

kubectl apply -f sc.yaml

5. Tweak Resource & JVM Settings​

Keep requests low for CI, limits high for prod bursts:

master:
  replicas: 1
  resources:
    limits:   { cpu: "4",  memory: "4Gi" }
    requests: { cpu: "500m", memory: "2Gi" }
  env:
    JAVA_OPTS: "-Xms1g -Xmx1g -Xmn512m"

worker:
  replicas: 3
  persistentVolumeClaim:
    enabled: true
    storageClassName: "nfs-storage"
    storage: "20Gi"
  resources:
    limits:   { cpu: "2",  memory: "4Gi" }
    requests: { cpu: "500m", memory: "2Gi" }

6. Deploy & Verify​

kubectl create ns dolphinscheduler
helm -n dolphinscheduler install ds -f values.yaml .

Watch the rollout:

kubectl -n dolphinscheduler get pods -w

All pods should be Running within 2-3 minutes.

7. Ingress & Custom Domain​

Expose the UI via Rancher’s Ingress:

ingress:
  enabled: true
  host: dolphinscheduler.tyzwkj.cn
  path: /dolphinscheduler

Add a local DNS entry:

# /etc/hosts (Linux/Mac) or C:\Windows\System32\drivers\etc\hosts (Win)
<K8S_NODE_IP> dolphinscheduler.tyzwkj.cn

Open http://dolphinscheduler.tyzwkj.cn/dolphinscheduler/ui/ and log in:

• User: admin
• Password: dolphinscheduler123

8. Where to Go Next​

• Scale workers horizontally using kubectl -n dolphinscheduler scale sts ds-dolphinscheduler-worker --replicas=5
• Enable Prometheus by flipping serviceMonitor.enabled: true
• Switch to external PostgreSQL/MySQL for real production workloads

Wrap-up​

By mirroring images, caching dependencies, and wiring NFS-backed volumes, we turned a flaky internet install into a repeatable, offline-capable deployment. Clone this repo, tweak values.yaml, and you’ll have a brand-new DolphinScheduler cluster in minutes—no matter how hostile the network.

If this saved you a few gray hairs, smash the button and share your own tweaks in the comments!

Continue reading...

 

Join 𝕋𝕄𝕋 on Telegram
Channel PREVIEW: