KnoAI 技术站

主题

K3s + HAMi vGPU + vLLM + Open WebUI 完整部署指南 ​

最后更新: 2026-08-22 集群版本: k3s v1.36.3+k3s1 | HAMi v2.9.0 | vLLM v0.8.4 | Calico v3.28.0

目录 ​

  1. 架构概览
  2. Phase 1: k3s 集群搭建
  3. Phase 2: Calico CNI 网络
  4. Phase 3: 私有镜像仓库
  5. Phase 4: NFS 模型存储
  6. Phase 5: HAMi vGPU 安装
  7. Phase 6: HAMi Per-Node 异构 GPU 配置
  8. Phase 7: vLLM 模型部署 (7 Pods)
  9. Phase 8: Open WebUI 集成
  10. Phase 9: Traefik Ingress 外网暴露
  11. Phase 10: 添加 GPU Worker 节点
  12. 验证与测试
  13. 排错记录
  14. 附录: 完整 Manifest 参考

1. 架构概览 ​

1.1 集群拓扑 ​

                          ┌──────────────────────────────────────────┐
                          │         Cloud NAT (22/80 only)           │
                          └────────────────────┬─────────────────────┘
                                               │ :80
                          ┌────────────────────▼─────────────────────┐
                          │  Server: 10-60-10-196 (117.50.188.237)  │
                          │  RTX 2080 8GB │ control-plane + worker   │
                          │  NFS Server │ Registry :30000            │
                          │  Traefik Ingress │ Open WebUI            │
                          │  HAMi Scheduler                         │
                          └────┬────────────┬────────────┬───────────┘
                               │            │            │
                    ┌──────────▼──┐  ┌──────▼──────┐     │
                    │ 10-60-38-105│  │10-60-156-30 │     │
                    │ (CPU only)  │  │ (CPU only)  │     │
                    └─────────────┘  └─────────────┘     │
                                                         │
                    ┌─────────────┐  ┌─────────────┐     │
                    │ 10-60-205-41│  │  10-60-18-8  │     │
                    │ RTX 3080 Ti │  │  RTX 3090    │     │
                    │ 12GB (2vGPU)│  │  24GB (4vGPU)│     │
                    └─────────────┘  └─────────────┘     │

1.2 节点清单 ​

节点名公网 IP内网 IP角色GPUvGPU 配置
10-60-10-196117.50.188.23710.60.10.196control-planeRTX 2080 8GB2 x 4GB
10-60-38-105117.50.215.18310.60.38.105worker无-
10-60-156-30117.50.190.15910.60.156.30worker无-
10-60-205-41117.50.185.6810.60.205.41worker (GPU)RTX 3080 Ti 12GB2 x 6GB
10-60-18-8117.50.213.12910.60.18.8worker (GPU)RTX 3090 24GB4 x 6GB

1.3 GPU 资源分配矩阵 ​

节点GPU总显存vGPU 切分数每 vGPU 显存可运行模型
10-60-10-196RTX 20808GB24GBQwen3-0.6B only
10-60-205-41RTX 3080 Ti12GB26GBQwen2.5-0.5B + Qwen3-0.6B
10-60-18-8RTX 309024GB46GB最多 4 个模型实例

关键约束: RTX 2080 的 4GB vGPU 无法运行 Qwen2.5-0.5B（模型 + KV cache 需 3.91GB + 116MB > 4GB），只能跑 Qwen3-0.6B。

1.4 当前运行的 7 个 vLLM 实例 ​

#部署名模型节点vGPU 显存Service
1vllm-qwen25-05bQwen2.5-0.5B10-60-18-8 (3090)6GBvllm-qwen25-05b:8001
2vllm-qwen3-06bQwen3-0.6B10-60-18-8 (3090)6GBvllm-qwen3-06b:8002
3vllm-qwen25-05b-3080tiQwen2.5-0.5B10-60-205-41 (3080Ti)6GBvllm-qwen25-05b-3080ti:8003
4vllm-qwen3-06b-3080tiQwen3-0.6B10-60-205-41 (3080Ti)6GBvllm-qwen3-06b-3080ti:8004
5vllm-qwen3-06b-2080Qwen3-0.6B10-60-10-196 (2080)4GBvllm-qwen3-06b-2080:8006
6vllm-qwen25-05b-3090-2Qwen2.5-0.5B10-60-18-8 (3090)6GBvllm-qwen25-05b-3090-2:8007
7vllm-qwen3-06b-3090-2Qwen3-0.6B10-60-18-8 (3090)6GBvllm-qwen3-06b-3090-2:8008

2. Phase 1: k3s 集群搭建 ​

2.1 思路 ​

  • 使用 k3s 轻量级 Kubernetes，国内镜像安装
  • 禁用 Flannel（后续使用 Calico），设置 flannel-backend: "none"
  • 禁用 network-policy（Calico 会接管）
  • kubeconfig 权限设为 0644，方便非 root 用户操作

2.2 安装 k3s ​

bash
# 所有节点执行
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

2.3 k3s 配置 ​

所有节点 -- 写入 /etc/rancher/k3s/config.yaml：

yaml
write-kubeconfig-mode: "0644"
flannel-backend: "none"
disable-network-policy: true

2.4 Server 节点启动 ​

bash
# 在 10-60-10-196 上
sudo systemctl enable --now k3s

2.5 Worker 节点加入 ​

思路: 由于国内无法直接下载 k3s 二进制，从 server 节点 scp 拷贝已安装的二进制文件。

bash
SERVER="ubuntu@117.50.188.237"
NEW_NODE="ubuntu@<新节点IP>"
SERVER_INTERNAL_IP="10.60.10.196"
TOKEN=$(ssh $SERVER "sudo cat /var/lib/rancher/k3s/server/node-token")

# 1. 拷贝 k3s 二进制
ssh $SERVER "scp /usr/local/bin/k3s $NEW_NODE:/tmp/k3s"
ssh $NEW_NODE "sudo install -m 755 /tmp/k3s /usr/local/bin/k3s"

# 2. 写入配置
ssh $NEW_NODE "sudo mkdir -p /etc/rancher/k3s"
ssh $NEW_NODE "cat << 'EOF' | sudo tee /etc/rancher/k3s/config.yaml
write-kubeconfig-mode: \"0644\"
flannel-backend: \"none\"
disable-network-policy: true
EOF"

# 3. 创建 systemd service
ssh $NEW_NODE "cat << 'SYSEOF' | sudo tee /etc/systemd/system/k3s-agent.service
[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
Wants=network-online.target
After=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
KillMode=process
Delegate=yes
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
Environment=K3S_URL=https://${SERVER_INTERNAL_IP}:6443
Environment=K3S_TOKEN=${TOKEN}
ExecStartPre=/sbin/modprobe br_netfilter
ExecStartPre=/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s agent
SYSEOF"

# 4. 启动
ssh $NEW_NODE "sudo systemctl daemon-reload && sudo systemctl enable --now k3s-agent"

# 5. 验证（在 server 上）
k3s kubectl get nodes

注意: 如果端口 10250 被占用，先 sudo pkill -9 k3s; sudo pkill -9 containerd 清理残留进程。

3. Phase 2: Calico CNI 网络 ​

3.1 思路 ​

  • k3s 默认 Flannel 不支持 NetworkPolicy，且跨节点 Pod 通信在某些场景下有坑
  • 选择 Calico v3.28.0，使用 VXLAN 模式（非 BGP，因为云环境不支持 BGP peering）
  • 所有 Calico 镜像预推送到本地 Registry，避免拉取超时

3.2 准备 Calico 镜像 ​

bash
CALICO_VERSION="v3.28.0"
REGISTRY="117.50.188.237:30000"
HUAWEI_MIRROR="swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io"

# 从华为云拉取
for img in cni node kube-controllers pod2daemon-flexvol csi node-driver-registrar; do
  docker pull $HUAWEI_MIRROR/calico/$img:$CALICO_VERSION
done

# 打标签并推送到本地仓库
for img in cni node kube-controllers pod2daemon-flexvol csi node-driver-registrar; do
  SRC="$HUAWEI_MIRROR/calico/$img:$CALICO_VERSION"
  DST="$REGISTRY/calico/$img:$CALICO_VERSION"
  docker tag $SRC $DST
  docker push $DST
done

3.3 部署 Calico ​

bash
# 下载 manifest
curl -sSL https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml -o calico.yaml

# 替换镜像地址
sed -i "s|docker.io/calico/|117.50.188.237:30000/calico/|g" calico.yaml

# 应用
k3s kubectl apply -f calico.yaml

# 等待就绪
k3s kubectl wait --for=condition=Ready pods -l k8s-app=calico-node -n kube-system --timeout=120s
k3s kubectl wait --for=condition=Ready pods -l k8s-app=calico-kube-controllers -n kube-system --timeout=120s

3.4 验证 ​

bash
k3s kubectl get pods -n kube-system | grep calico
ip addr show | grep -E "cali|tunl|vxlan"

# 跨节点 Pod 通信测试
k3s kubectl run test-pod --image=busybox --restart=Never --rm -it -- sh -c "ping -c 2 8.8.8.8"

3.5 registries.yaml（所有节点） ​

思路: 所有镜像仓库 mirror 都指向本地 Registry，确保拉取速度和国内可用性。

yaml
# /etc/rancher/k3s/registries.yaml
mirrors:
  docker.io:
    endpoint:
      - "http://117.50.188.237:30000"
      - "https://registry.cn-hangzhou.aliyuncs.com"
  registry.k8s.io:
    endpoint:
      - "http://117.50.188.237:30000"
      - "https://registry.cn-hangzhou.aliyuncs.com"
  gcr.io:
    endpoint:
      - "http://117.50.188.237:30000"
      - "https://registry.cn-hangzhou.aliyuncs.com"
  ghcr.io:
    endpoint:
      - "http://117.50.188.237:30000"
      - "https://registry.cn-hangzhou.aliyuncs.com"
  quay.io:
    endpoint:
      - "http://117.50.188.237:30000"
      - "https://registry.cn-hangzhou.aliyuncs.com"
  registry.cn-hangzhou.aliyuncs.com:
    endpoint:
      - "http://117.50.188.237:30000"
  "117.50.188.237:30000":
    endpoint:
      - "http://117.50.188.237:30000"
bash
# 写入后重启 k3s
sudo systemctl restart k3s

注意: 每个新节点加入前都必须写入此文件，否则系统 Pod（pause、coredns 等）拉取超时。

4. Phase 3: 私有镜像仓库 ​

4.1 思路 ​

  • 国内环境无法直接拉取 Docker Hub / GHCR / gcr.io 镜像
  • 在 server 节点上部署 Docker Registry，端口 30000
  • 所有需要的镜像预先拉取并推送到本地 Registry

4.2 部署 Registry ​

bash
sudo mkdir -p /opt/registry

docker run -d \
  --name docker-registry \
  --restart=always \
  -p 30000:5000 \
  -v /opt/registry:/var/lib/registry \
  -e REGISTRY_STORAGE_DELETE_ENABLED=true \
  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/registry:latest

4.3 推送镜像 ​

bash
REGISTRY="localhost:30000"
CTR="k3s ctr"

# 对每个需要的镜像执行 tag + push
push_image() {
  local IMG="$1"
  $CTR images tag "$IMG" "${REGISTRY}/${IMG}"
  $CTR images push --plain-http "${REGISTRY}/${IMG}"

  # 同时推送无前缀版本（供 k3s mirror 使用）
  local SHORT="${IMG#docker.io/}"
  SHORT="${SHORT#ghcr.io/}"
  $CTR images tag "$IMG" "${REGISTRY}/${SHORT}"
  $CTR images push --plain-http "${REGISTRY}/${SHORT}"
}

# 核心镜像
push_image "docker.io/vllm/vllm-openai:v0.8.4"
push_image "docker.io/projecthami/hami:v2.9.0"
push_image "docker.io/rancher/klipper-lb:v0.4.17"
push_image "docker.io/rancher/klipper-helm:v0.13.3-build20260727"
push_image "docker.io/rancher/local-path-provisioner:v0.0.36"
push_image "docker.io/rancher/mirrored-coredns-coredns:1.14.6"
push_image "docker.io/rancher/mirrored-library-traefik:3.7.8"
push_image "docker.io/rancher/mirrored-metrics-server:v0.9.0"
push_image "docker.io/rancher/mirrored-pause:3.10.2"
push_image "docker.io/rancher/mirrored-pause:3.6"
push_image "ghcr.io/open-webui/open-webui:main"
push_image "registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.36.3"
push_image "docker.io/liangjw/kube-webhook-certgen:v1.1.1"
push_image "docker.io/jettech/kube-webhook-certgen:v1.5.2"

4.4 验证 ​

bash
curl -s http://117.50.188.237:30000/v2/_catalog | python3 -m json.tool
du -sh /opt/registry/   # ~11GB

5. Phase 4: NFS 模型存储 ​

5.1 思路 ​

  • 模型文件 (~2GB) 需要在所有 GPU 节点共享访问
  • 使用 NFS 共享，避免每个节点重复拷贝模型
  • 采用静态 PV/PVC 方案（不依赖 NFS provisioner，省去额外镜像）
  • NFS Server 部署在 server 节点 (10.60.10.196)

5.2 NFS Server 配置 ​

bash
# 安装
sudo apt-get install -y nfs-kernel-server

# 配置导出
sudo tee /etc/exports > /dev/null << 'EOF'
/model/ModelScope *(rw,sync,no_subtree_check,no_root_squash)
EOF

# 启动
sudo systemctl enable --now nfs-kernel-server
sudo exportfs -ra

5.3 模型文件目录 ​

/model/ModelScope/Qwen/
+-- Qwen2.5-0.5B-Instruct/    # ~1.1GB
+-- Qwen3-0.6B/               # ~1.2GB

5.4 NFS Client（所有 Worker 节点） ​

bash
sudo apt-get install -y nfs-common

# 验证挂载
sudo mount -t nfs 10.60.10.196:/model/ModelScope /mnt
ls /mnt/
sudo umount /mnt

5.5 创建 PV/PVC ​

yaml
# nfs-pv-pvc.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-modelscope
spec:
  capacity:
    storage: 200Gi
  accessModes:
    - ReadWriteMany
  storageClassName: nfs-modelscope
  persistentVolumeReclaimPolicy: Retain
  nfs:
    path: /model/ModelScope
    server: 10.60.10.196
    readOnly: false
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-modelscope
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: nfs-modelscope
  volumeName: nfs-modelscope
  resources:
    requests:
      storage: 200Gi
bash
k3s kubectl apply -f nfs-pv-pvc.yaml
k3s kubectl get pv,pvc nfs-modelscope

6. Phase 5: HAMi vGPU 安装 ​

6.1 思路 ​

  • RTX 2080 (8GB) / RTX 3080 Ti (12GB) / RTX 3090 (24GB) 均为消费级显卡，不支持 NVIDIA MIG
  • HAMi 的 hami-core 模式通过 libvgpu.so CUDA Hook 实现软件 vGPU，无需 MIG 硬件
  • 核心原理: LD_PRELOAD 注入 libvgpu.so，拦截 cudaMalloc 等 API，实现显存隔离
  • 安装方式: Helm Chart，配合自定义 values.yaml

6.2 HAMi 工作原理 ​

用户 Pod 请求 GPU
       |
       v
+----------------+     +--------------------+     +------------------+
| hami-scheduler |---->| hami-device-plugin |---->| libvgpu.so (注入)|
| (调度决策)      |     | (设备注册+分配)     |     | (运行时资源限制) |
+----------------+     +--------------------+     +------------------+

调度流程:

  1. device-plugin 启动 -> 探测物理 GPU -> 写入 node annotation (hami.io/node-nvidia-register)
  2. 用户 Pod 声明 nvidia.com/gpu: 1 + nvidia.com/gpumem: 6000
  3. hami-scheduler filter 阶段评估节点剩余资源 -> bind Pod 到节点
  4. device-plugin Allocate 阶段 -> 注入 libvgpu.so + 设置 CUDA_DEVICE_MEMORY_LIMIT_0
  5. 容器启动后只能看到/使用被分配的显存量

6.3 标记 GPU 节点 ​

bash
# 每个 GPU 节点都需要标记
k3s kubectl label node 10-60-10-196 gpu=on --overwrite
k3s kubectl label node 10-60-205-41 gpu=on --overwrite
k3s kubectl label node 10-60-18-8 gpu=on --overwrite

6.4 Helm 安装 ​

bash
helm repo add hami https://project-hami.github.io/HAMi/
helm repo update hami

创建 hami-values.yaml：

yaml
global:
  imageTag: "v2.9.0"
  gpuHookPath: /usr/local

devicePlugin:
  deviceSplitCount: 4          # 默认值（会被 nodeconfig 覆盖）
  deviceMemoryScaling: 1
  deviceCoreScaling: 1
  nvidiaNodeSelector:
    gpu: "on"
  nvidiaDriverRoot: "/"        # Ubuntu apt 驱动根目录
  runtimeClassName: nvidia
  image:
    registry: "117.50.188.237:30000"
    repository: "projecthami/hami"
    tag: "v2.9.0"
    pullPolicy: IfNotPresent
    pullSecrets: []
  monitor:
    image:
      registry: "117.50.188.237:30000"
      repository: "projecthami/hami"
      tag: "v2.9.0"

scheduler:
  nodeName: "10-60-10-196"
  kubeScheduler:
    image:
      registry: "117.50.188.237:30000"
      repository: "google_containers/kube-scheduler"
      tag: "v1.36.3"
      pullPolicy: IfNotPresent
      pullSecrets: []
  extender:
    image:
      registry: "117.50.188.237:30000"
      repository: "projecthami/hami"
      tag: "v2.9.0"
      pullPolicy: IfNotPresent
      pullSecrets: []
bash
helm install hami hami/hami \
  -n hami-system --create-namespace -f hami-values.yaml

6.5 Patch DaemonSet（Ubuntu 驱动兼容） ​

思路: Ubuntu apt 安装的 NVIDIA 驱动库在 /usr/lib/x86_64-linux-gnu/，HAMi 的 monitor 容器默认无法访问。需要:

  1. 添加 LD_LIBRARY_PATH 环境变量指向驱动库路径
  2. 设置 monitor 容器为 privileged: true
  3. 挂载 /dev 目录
bash
k3s kubectl patch ds hami-device-plugin -n hami-system --type=json -p='[
  {"op":"add","path":"/spec/template/spec/containers/1/env/-","value":{"name":"LD_LIBRARY_PATH","value":"/driver-root/usr/lib/x86_64-linux-gnu:/driver-root/usr/lib32:/usr/local/vgpu"}},
  {"op":"add","path":"/spec/template/spec/containers/1/volumeMounts/-","value":{"name":"driver-root","mountPath":"/driver-root","readOnly":true}},
  {"op":"replace","path":"/spec/template/spec/containers/1/securityContext","value":{"privileged":true,"allowPrivilegeEscalation":true}},
  {"op":"add","path":"/spec/template/spec/containers/1/volumeMounts/-","value":{"name":"dev","mountPath":"/dev"}},
  {"op":"add","path":"/spec/template/spec/volumes/-","value":{"name":"dev","hostPath":{"path":"/dev"}}}
]'

6.6 验证 HAMi ​

bash
# 检查 Pod 状态
k3s kubectl get pods -n hami-system
# 预期: hami-device-plugin-xxx 2/2 Running, hami-scheduler-xxx 2/2 Running

# 检查 vGPU 资源
k3s kubectl get nodes -o json | python3 -c "
import json, sys
data = json.load(sys.stdin)
for node in data['items']:
    alloc = node['status'].get('allocatable', {})
    gpu = alloc.get('nvidia.com/gpu', '0')
    mem = alloc.get('nvidia.com/gpumem', '0')
    if gpu != '0':
        print(f\"{node['metadata']['name']}: GPU={gpu}, VRAM={mem}MB\")
"

7. Phase 6: HAMi Per-Node 异构 GPU 配置 ​

7.1 问题 ​

HAMi Helm 安装的 deviceSplitCount 是全局统一的，但我们的集群有 3 种不同的 GPU:

GPU显存期望切分每 vGPU
RTX 309024GB4 份6GB
RTX 3080 Ti12GB2 份6GB
RTX 20808GB2 份4GB

如果用全局 deviceSplitCount=4，2080 会被切成 4x2GB（太小）；如果用 2，3090 只切 2x12GB（浪费）。

7.2 解决方案: ConfigMap nodeconfig ​

HAMi 的 hami-device-plugin ConfigMap 支持 nodeconfig 数组，为每个节点单独配置切分策略。

bash
# 编辑 ConfigMap
k3s kubectl edit configmap hami-device-plugin -n hami-system

写入以下 config.json:

json
{
  "nodeconfig": [
    {
      "name": "10-60-18-8",
      "operatingmode": "hami-core",
      "devicememoryscaling": 1,
      "devicesplitcount": 4,
      "preconfigureddevicememory": 0,
      "enablenumatopology": false,
      "migstrategy": "none",
      "filterdevices": { "uuid": [], "index": [] },
      "enablegetpreferredallocation": false
    },
    {
      "name": "10-60-205-41",
      "operatingmode": "hami-core",
      "devicememoryscaling": 1,
      "devicesplitcount": 2,
      "preconfigureddevicememory": 0,
      "enablenumatopology": false,
      "migstrategy": "none",
      "filterdevices": { "uuid": [], "index": [] },
      "enablegetpreferredallocation": false
    },
    {
      "name": "10-60-10-196",
      "operatingmode": "hami-core",
      "devicememoryscaling": 1,
      "devicesplitcount": 2,
      "preconfigureddevicememory": 0,
      "enablenumatopology": false,
      "migstrategy": "none",
      "filterdevices": { "uuid": [], "index": [] },
      "enablegetpreferredallocation": false
    }
  ]
}

7.3 应用配置 ​

bash
# 修改 ConfigMap 后，重启所有 device-plugin Pod 使配置生效
k3s kubectl delete pods -n hami-system -l app.kubernetes.io/component=hami-device-plugin

# 同时重启 scheduler（清除缓存的旧节点信息）
k3s kubectl delete pods -n hami-system -l app.kubernetes.io/component=hami-scheduler

# 验证各节点的 vGPU 注册信息
k3s kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}: {.metadata.annotations.hami\.io/node-nvidia-register}{"\n"}{end}'

注意: 新 GPU 节点加入后，需要在 ConfigMap 的 nodeconfig 数组中添加对应条目，然后重启 device-plugin。

8. Phase 7: vLLM 模型部署 (7 Pods) ​

8.1 通用 vLLM 参数说明 ​

所有 vLLM Pod 共享以下启动参数:

参数值说明
--devicecuda使用 GPU
--dtypefloat16RTX 2080 不支持 bfloat16 (compute 7.5 < 8.0)，统一用 float16
--max-model-len2048最大上下文长度
--enforce-eager-禁用 CUDA graph（节省显存）
--max-num-seqs4限制并发序列数（节省 KV cache）
--gpu-memory-utilization0.95允许使用 95% 显存
--swap-space0不使用 CPU swap

8.2 6GB vGPU 部署模板 (3090 / 3080 Ti) ​

以下以 vllm-qwen25-05b (3090 节点) 为例。其他 6GB vGPU 部署只需修改:

  • metadata.name / labels
  • nodeSelector 的目标节点
  • Service.port / Service.name
yaml
# vllm-6gb-template.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm-qwen25-05b
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vllm-qwen25-05b
  template:
    metadata:
      labels:
        app: vllm-qwen25-05b
    spec:
      schedulerName: hami-scheduler
      nodeSelector:
        kubernetes.io/hostname: "10-60-18-8"    # <-- 指定目标 GPU 节点
      containers:
      - name: vllm
        image: 117.50.188.237:30000/vllm/vllm-openai:v0.8.4
        imagePullPolicy: IfNotPresent
        args:
        - --model
        - /models/Qwen/Qwen2.5-0.5B-Instruct   # <-- 模型路径
        - --served-model-name
        - qwen2.5-0.5b                          # <-- API 模型名
        - --port
        - "8000"
        - --device
        - cuda
        - --dtype
        - float16
        - --max-model-len
        - "2048"
        - --enforce-eager
        - --max-num-seqs
        - "4"
        - --gpu-memory-utilization
        - "0.95"
        - --swap-space
        - "0"
        ports:
        - containerPort: 8000
        env:
        - name: HF_ENDPOINT
          value: "https://hf-mirror.com"
        resources:
          requests:
            cpu: "500m"
            memory: "2Gi"
            nvidia.com/gpu: "1"
            nvidia.com/gpumem: "6k"           # 6000MB 绝对显存
          limits:
            memory: "6Gi"
            nvidia.com/gpu: "1"
            nvidia.com/gpumem: "6k"
        volumeMounts:
        - name: models
          mountPath: /models
          readOnly: true
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 60
          periodSeconds: 10
          failureThreshold: 3
      volumes:
      - name: models
        persistentVolumeClaim:
          claimName: nfs-modelscope
---
apiVersion: v1
kind: Service
metadata:
  name: vllm-qwen25-05b
spec:
  type: ClusterIP
  selector:
    app: vllm-qwen25-05b
  ports:
  - port: 8001                               # <-- Service 端口
    targetPort: 8000
    protocol: TCP

8.3 所有 6GB vGPU 部署清单 ​

部署名nodeSelector模型Service port
vllm-qwen25-05b10-60-18-8Qwen2.5-0.5B-Instruct8001
vllm-qwen3-06b10-60-18-8Qwen3-0.6B8002
vllm-qwen25-05b-3080ti10-60-205-41Qwen2.5-0.5B-Instruct8003
vllm-qwen3-06b-3080ti10-60-205-41Qwen3-0.6B8004
vllm-qwen25-05b-3090-210-60-18-8Qwen2.5-0.5B-Instruct8007
vllm-qwen3-06b-3090-210-60-18-8Qwen3-0.6B8008

每个部署只需复制模板，修改上述 4 个字段。

8.4 4GB vGPU 部署 (RTX 2080) ​

限制: 4GB vGPU 只能运行 Qwen3-0.6B，无法运行 Qwen2.5-0.5B (OOM)。

yaml
# vllm-qwen3-2080.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm-qwen3-06b-2080
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vllm-qwen3-06b-2080
  template:
    metadata:
      labels:
        app: vllm-qwen3-06b-2080
    spec:
      schedulerName: hami-scheduler
      nodeSelector:
        kubernetes.io/hostname: "10-60-10-196"
      containers:
      - name: vllm
        image: 117.50.188.237:30000/vllm/vllm-openai:v0.8.4
        imagePullPolicy: IfNotPresent
        args:
        - --model
        - /models/Qwen/Qwen3-0.6B
        - --served-model-name
        - qwen3-0.6b
        - --port
        - "8000"
        - --device
        - cuda
        - --dtype
        - float16
        - --max-model-len
        - "2048"
        - --enforce-eager
        - --max-num-seqs
        - "4"
        - --gpu-memory-utilization
        - "0.95"
        - --swap-space
        - "0"
        ports:
        - containerPort: 8000
        env:
        - name: HF_ENDPOINT
          value: "https://hf-mirror.com"
        resources:
          requests:
            cpu: "500m"
            memory: "2Gi"
            nvidia.com/gpu: "1"
            nvidia.com/gpumem: "4k"           # 4000MB (2080 的 vGPU)
          limits:
            memory: "4Gi"
            nvidia.com/gpu: "1"
            nvidia.com/gpumem: "4k"
        volumeMounts:
        - name: models
          mountPath: /models
          readOnly: true
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 60
          periodSeconds: 10
          failureThreshold: 3
      volumes:
      - name: models
        persistentVolumeClaim:
          claimName: nfs-modelscope
---
apiVersion: v1
kind: Service
metadata:
  name: vllm-qwen3-06b-2080
spec:
  type: ClusterIP
  selector:
    app: vllm-qwen3-06b-2080
  ports:
  - port: 8006
    targetPort: 8000
    protocol: TCP

8.5 部署命令 ​

bash
# 按顺序部署所有 7 个实例
k3s kubectl apply -f vllm-qwen25-05b.yaml          # 3090, :8001
k3s kubectl apply -f vllm-qwen3-06b.yaml           # 3090, :8002
k3s kubectl apply -f vllm-qwen25-05b-3080ti.yaml   # 3080Ti, :8003
k3s kubectl apply -f vllm-qwen3-06b-3080ti.yaml    # 3080Ti, :8004
k3s kubectl apply -f vllm-qwen3-06b-2080.yaml      # 2080, :8006
k3s kubectl apply -f vllm-qwen25-05b-3090-2.yaml   # 3090, :8007
k3s kubectl apply -f vllm-qwen3-06b-3090-2.yaml    # 3090, :8008

# 验证
k3s kubectl get pods -o wide | grep vllm

9. Phase 8: Open WebUI 集成 ​

9.1 思路 ​

  • Open WebUI 通过 OPENAI_API_BASE_URLS 环境变量配置后端 vLLM 端点
  • 多个 URL 用分号分隔，指向所有 7 个 ClusterIP Service
  • 当配置多个 URL 时，必须使用 OPENAI_API_KEYS（复数） 并提供等数量的占位 key
  • 禁用 RAG（需要下载嵌入模型，国内不稳定）

9.2 Deployment ​

yaml
# open-webui.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: open-webui
spec:
  replicas: 1
  selector:
    matchLabels:
      app: open-webui
  template:
    metadata:
      labels:
        app: open-webui
    spec:
      containers:
      - name: open-webui
        image: 117.50.188.237:30000/open-webui/open-webui:main
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          name: http
        env:
        - name: OPENAI_API_BASE_URLS
          value: "http://vllm-qwen25-05b:8001/v1;http://vllm-qwen3-06b:8002/v1;http://vllm-qwen25-05b-3080ti:8003/v1;http://vllm-qwen3-06b-3080ti:8004/v1;http://vllm-qwen3-06b-2080:8006/v1;http://vllm-qwen25-05b-3090-2:8007/v1;http://vllm-qwen3-06b-3090-2:8008/v1"
        - name: OPENAI_API_KEYS
          value: "not-needed;not-needed;not-needed;not-needed;not-needed;not-needed;not-needed"
        - name: WEBUI_AUTH
          value: "false"
        - name: HF_ENDPOINT
          value: "https://hf-mirror.com"
        - name: TRANSFORMERS_OFFLINE
          value: "0"
        - name: ENABLE_RAG_WEB_SEARCH
          value: "false"
        - name: RAG_EMBEDDING_MODEL
          value: ""
        - name: CHUNK_SIZE
          value: "0"
        volumeMounts:
        - name: data
          mountPath: /app/backend/data
      volumes:
      - name: data
        hostPath:
          path: /opt/open-webui-data
          type: DirectoryOrCreate
---
apiVersion: v1
kind: Service
metadata:
  name: open-webui
spec:
  selector:
    app: open-webui
  ports:
  - port: 8080
    targetPort: 8080
    nodePort: 30080
  type: NodePort
bash
sudo mkdir -p /opt/open-webui-data
sudo chmod 777 /opt/open-webui-data
k3s kubectl apply -f open-webui.yaml

踩坑: 如果 OPENAI_API_BASE_URLS 有 7 个 URL 但只提供 OPENAI_API_KEY=not-needed（单数），Open WebUI 启动时会报 index out of range。必须用 OPENAI_API_KEYS（复数）并用分号提供 7 个占位 key。

10. Phase 9: Traefik Ingress 外网暴露 ​

10.1 思路 ​

  • 云厂商 NAT 仅放行 22/80 端口，NodePort (30001/30002/30080) 无法从外部直接访问
  • k3s 内置 Traefik，使用 IngressRoute CRD 暴露服务
  • 路径前缀: /api/qwen25 -> vllm-qwen25-05b, /api/qwen3 -> vllm-qwen3-06b, / -> open-webui

10.2 Middleware (路径剥离) ​

yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: strip-qwen25
spec:
  stripPrefix:
    prefixes:
      - /api/qwen25
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: strip-qwen3
spec:
  stripPrefix:
    prefixes:
      - /api/qwen3

10.3 IngressRoute ​

yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: vllm-ingress
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix("/api/qwen25")
    kind: Rule
    services:
    - name: vllm-qwen25-05b
      port: 8001
    middlewares:
    - name: strip-qwen25
  - match: PathPrefix("/api/qwen3")
    kind: Rule
    services:
    - name: vllm-qwen3-06b
      port: 8002
    middlewares:
    - name: strip-qwen3
  - match: PathPrefix("/")
    kind: Rule
    services:
    - name: open-webui
      port: 8080
bash
k3s kubectl apply -f ingress.yaml

10.4 外网访问地址 ​

服务URL
Open WebUIhttp://117.50.188.237/
Qwen2.5 APIhttp://117.50.188.237/api/qwen25/v1/models
Qwen3 APIhttp://117.50.188.237/api/qwen3/v1/models

11. Phase 10: 添加 GPU Worker 节点 ​

11.1 思路 ​

新 GPU 节点加入集群需要:

  1. 加入 k3s 集群（拷贝二进制 + token）
  2. 安装 NVIDIA 驱动
  3. 配置 registries.yaml（镜像加速）
  4. 安装 nfs-common（模型文件挂载）
  5. 标记 gpu=on 标签
  6. 在 HAMi ConfigMap nodeconfig 中添加该节点的切分配置
  7. 重启 HAMi device-plugin（自动在新节点部署 DaemonSet Pod）
  8. 部署 vLLM Pod（指定 nodeSelector）

11.2 加入 k3s 集群 ​

参见 Phase 1 的 Worker 节点加入脚本。

11.3 安装 NVIDIA 驱动 ​

bash
# 在新节点上
sudo apt-get update
sudo apt-get install -y nvidia-driver-535
sudo reboot

# 验证
nvidia-smi

11.4 配置节点环境 ​

bash
# registries.yaml
sudo mkdir -p /etc/rancher/k3s
sudo tee /etc/rancher/k3s/registries.yaml << 'EOF'
mirrors:
  docker.io:
    endpoint:
      - "http://117.50.188.237:30000"
      - "https://registry.cn-hangzhou.aliyuncs.com"
  registry.k8s.io:
    endpoint:
      - "http://117.50.188.237:30000"
  ghcr.io:
    endpoint:
      - "http://117.50.188.237:30000"
  "117.50.188.237:30000":
    endpoint:
      - "http://117.50.188.237:30000"
EOF

# NFS client
sudo apt-get install -y nfs-common

# 重启 k3s-agent
sudo systemctl restart k3s-agent

11.5 标记 GPU 节点 ​

bash
k3s kubectl label node <node-name> gpu=on --overwrite

11.6 添加 HAMi Per-Node 配置 ​

bash
k3s kubectl edit configmap hami-device-plugin -n hami-system
# 在 nodeconfig 数组中添加:
# {
#   "name": "<node-name>",
#   "operatingmode": "hami-core",
#   "devicesplitcount": <N>,
#   "devicememoryscaling": 1,
#   "preconfigureddevicememory": 0
# }

# 重启生效
k3s kubectl delete pods -n hami-system -l app.kubernetes.io/component=hami-device-plugin
k3s kubectl delete pods -n hami-system -l app.kubernetes.io/component=hami-scheduler

11.7 硬件坑: 10-60-18-8 的"幽灵 GPU" ​

现象: 新节点 10-60-18-8 上 nvidia-smi 显示 2 张 RTX 3090，但 GPU 0 (PCIe 00:03.0) 无法使用：固件版本 N/A，UUID 全为 ?。

原因: 这是一个 VM GPU 透传的"幽灵设备"，PCI 设备枚举时存在但驱动无法绑定。

处理: 仅使用 GPU 1 (PCIe 00:04.0)，即只有 1 张可用 RTX 3090。HAMi devicesplitcount=4 只切 1 张卡，不影响。

12. 验证与测试 ​

12.1 集群状态检查 ​

bash
# 节点
k3s kubectl get nodes -o wide

# HAMi 系统
k3s kubectl get pods -n hami-system -o wide

# 所有 vLLM Pod
k3s kubectl get pods -o wide | grep vllm

# Open WebUI
k3s kubectl get pods -o wide | grep open-webui

# Services
k3s kubectl get svc | grep -E "vllm|open-webui"

12.2 GPU 推理测试（内网 ClusterIP） ​

bash
# 测试每个端点
for SVC in "vllm-qwen25-05b:8001" "vllm-qwen3-06b:8002" \
           "vllm-qwen25-05b-3080ti:8003" "vllm-qwen3-06b-3080ti:8004" \
           "vllm-qwen3-06b-2080:8006" \
           "vllm-qwen25-05b-3090-2:8007" "vllm-qwen3-06b-3090-2:8008"; do
  echo "=== Testing $SVC ==="
  curl -s --max-time 10 -X POST http://$SVC/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"qwen3-0.6b","messages":[{"role":"user","content":"Hello!"}],"max_tokens":15}' \
    | python3 -c "import json,sys; d=json.load(sys.stdin); \
      print('  Reply:', d['choices'][0]['message']['content'][:50])" 2>/dev/null \
    || echo "  FAILED"
done

12.3 外网 API 测试 ​

bash
# Open WebUI (browser)
curl -s http://117.50.188.237/ | head -5

# Qwen2.5 API (through Traefik)
curl -s http://117.50.188.237/api/qwen25/v1/models | python3 -m json.tool

# Qwen3 API
curl -s http://117.50.188.237/api/qwen3/v1/models | python3 -m json.tool

# Chat completion via public IP
curl -s http://117.50.188.237/api/qwen25/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen2.5-0.5b","messages":[{"role":"user","content":"1+1=?"}]}'

12.4 GPU 使用率检查 ​

bash
# 在每个 GPU 节点上执行
nvidia-smi

12.5 快速检查脚本 ​

bash
#!/bin/bash
# cluster-status.sh -- 一键检查集群状态

echo "=== Nodes ==="
k3s kubectl get nodes -o wide

echo ""
echo "=== HAMi System ==="
k3s kubectl get pods -n hami-system -o wide

echo ""
echo "=== vLLM Pods ==="
k3s kubectl get pods -o wide | grep vllm

echo ""
echo "=== Open WebUI ==="
k3s kubectl get pods -o wide | grep open-webui

echo ""
echo "=== Services ==="
k3s kubectl get svc | grep -E "vllm|open-webui|traefik"

echo ""
echo "=== GPU Resources ==="
k3s kubectl get nodes -o json | python3 -c "
import json, sys
data = json.load(sys.stdin)
for node in data['items']:
    alloc = node['status'].get('allocatable', {})
    gpu = alloc.get('nvidia.com/gpu', '0')
    mem = alloc.get('nvidia.com/gpumem', '0')
    if gpu != '0':
        print(f\"  {node['metadata']['name']}: GPU={gpu}, VRAM={mem}MB\")
"

echo ""
echo "=== Chat Tests ==="
for CLUSTER_IP in "10.43.73.160:8001" "10.43.172.124:8002" "10.43.177.97:8003" \
                  "10.43.134.169:8004" "10.43.93.230:8006" \
                  "10.43.37.132:8007" "10.43.188.227:8008"; do
  echo -n "  $CLUSTER_IP: "
  curl -s --max-time 10 -X POST http://$CLUSTER_IP/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"qwen3-0.6b","messages":[{"role":"user","content":"Hi"}],"max_tokens":10}' \
    | python3 -c "import json,sys; d=json.load(sys.stdin); \
      print('OK -', d['choices'][0]['message']['content'][:30])" 2>/dev/null \
    || echo "FAILED"
done

13. 排错记录 ​

13.1 HAMi 相关 ​

问题原因解决方案
device-plugin CrashLoopBackOff: NVML ERROR_LIBRARY_NOT_FOUNDUbuntu apt 驱动库路径不在容器搜索路径nvidiaDriverRoot: "/" + LD_LIBRARY_PATH=/driver-root/usr/lib/x86_64-linux-gnu
monitor 容器: Driver Not Loadedmonitor 容器非特权且无 /dev 访问Patch DaemonSet: privileged: true + 挂载 /dev
调度: node unregistered强制删除 Pod 后 HAMi 节点注册过期重启 device-plugin Pod 触发重新注册
Helm 安装失败: cannot overwrite table with non tableimage 字段用了简单字符串而非结构化对象改用 {registry, repository, tag, pullPolicy, pullSecrets} 结构
device-plugin 重启后 node handshake: Deletedscheduler 缓存过期同时重启 hami-scheduler + device-plugin
全局 deviceSplitCount 无法满足异构 GPU3090/3080Ti/2080 需要不同切分数使用 ConfigMap nodeconfig 按节点设置
滚动更新时旧 Pod 占 GPU 槽位，新 Pod PendingHAMi GPU 资源未释放kubectl scale rs <old-rs> --replicas=0

13.2 vLLM 相关 ​

问题原因解决方案
Bfloat16 not supportedRTX 2080 计算能力 7.5 < 8.0--dtype half
exec: "python" not foundvLLM 旧版镜像用 command 而非 ENTRYPOINT改用 args（v0.8.4 已修复）
Qwen3 架构不识别vLLM v0.6.1 太旧升级到 v0.8.4
UnspecifiedPlatform (CPU 模式)importlib.metadata.version("vllm") 返回 "unspecified"注入 .pth 补丁拦截版本号
CPU 模式 OOM4GB 节点同时跑两个 float32 实例单实例 + --swap-space 1 --max-num-seqs 4
provide the model as a positional argumentvLLM v0.8.4 起 serve 禁止 --model 选项模型路径作为位置参数
2080 4GB vGPU OOM (CUDA)Qwen2.5-0.5B + KV cache > 4GB2080 只部署 Qwen3-0.6B

13.3 网络/镜像相关 ​

问题原因解决方案
系统镜像拉取超时 (rancher/*)Docker Hub 国内不通华为云 SWR 手动拉取 + ctr images tag
k3s 重启后 pause 超时registries.yaml 缺少 registry.k8s.io mirror添加 mirror
Calico Init 卡在新节点registries.yaml 未在新节点配置配置后删除 Pod 重建
svclb-traefik ImagePullBackOffklipper-lb 版本不匹配确认正确版本 v0.4.17
NodePort 外网不可达云厂商 NAT 仅放行 22/80Traefik IngressRoute
config.toml.tmpl 导致默认配置丢失containerd 自定义模板完全覆盖删除 tmpl，仅用 registries.yaml

13.4 Open WebUI ​

问题原因解决方案
OPENAI_API_KEYS index out of range多个 URL 但只提供 1 个 key用 OPENAI_API_KEYS（复数）+ 分号分隔等数量占位 key
huggingface 连不上需下载 sentence-transformers 嵌入模型HF_ENDPOINT=https://hf-mirror.com + 禁用 RAG
NFS provisioner 镜像拉取超时registry.k8s.io 国内不通改用静态 NFS PV/PVC

13.5 硬件 ​

问题原因解决方案
新节点 GPU 0 (00:03.0) nvidia-smi 不可见VM GPU 透传问题，PCI 设备存在但固件 N/A、UUID 全为 ?仅使用 GPU 1 (00:04.0)
新节点 k3s-agent 启动失败: bind: address already in use (10250)残留 k3s 进程sudo pkill -9 k3s; sudo pkill -9 containerd
HAMi device-plugin CrashLoopBackOffGPU detached，nvidia 内核模块未加载GPU 重挂后自动恢复

14. 附录: 完整 Manifest 参考 ​

14.1 HAMi Scheduler ConfigMap ​

yaml
# hami-scheduler-device ConfigMap (hami-system namespace)
nvidia:
  resourceCountName: nvidia.com/gpu
  resourceMemoryName: nvidia.com/gpumem
  resourceMemoryPercentageName: nvidia.com/gpumem-percentage
  resourceCoreName: nvidia.com/gpucores
  resourcePriorityName: nvidia.com/priority
  overwriteEnv: false
  defaultMemory: 0
  defaultCores: 0
  defaultGPUNum: 1
  preConfiguredDeviceMemory: 0
  memoryFactor: 1
  deviceSplitCount: 4
  deviceMemoryScaling: 1
  deviceCoreScaling: 1
  enableNumaTopology: false
  gpuCorePolicy: default
  libCudaLogLevel: 1
  runtimeClassName: "nvidia"

14.2 HAMi Device-Plugin ConfigMap ​

json
{
  "nodeconfig": [
    {
      "name": "10-60-18-8",
      "operatingmode": "hami-core",
      "devicememoryscaling": 1,
      "devicesplitcount": 4,
      "preconfigureddevicememory": 0,
      "enablenumatopology": false,
      "migstrategy": "none",
      "filterdevices": { "uuid": [], "index": [] },
      "enablegetpreferredallocation": false
    },
    {
      "name": "10-60-205-41",
      "operatingmode": "hami-core",
      "devicememoryscaling": 1,
      "devicesplitcount": 2,
      "preconfigureddevicememory": 0,
      "enablenumatopology": false,
      "migstrategy": "none",
      "filterdevices": { "uuid": [], "index": [] },
      "enablegetpreferredallocation": false
    },
    {
      "name": "10-60-10-196",
      "operatingmode": "hami-core",
      "devicememoryscaling": 1,
      "devicesplitcount": 2,
      "preconfigureddevicememory": 0,
      "enablenumatopology": false,
      "migstrategy": "none",
      "filterdevices": { "uuid": [], "index": [] },
      "enablegetpreferredallocation": false
    }
  ]
}

14.3 NFS PV/PVC ​

yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-modelscope
spec:
  capacity:
    storage: 200Gi
  accessModes:
    - ReadWriteMany
  storageClassName: nfs-modelscope
  persistentVolumeReclaimPolicy: Retain
  nfs:
    path: /model/ModelScope
    server: 10.60.10.196
    readOnly: false
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-modelscope
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: nfs-modelscope
  volumeName: nfs-modelscope
  resources:
    requests:
      storage: 200Gi

14.4 Traefik IngressRoute ​

yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: strip-qwen25
spec:
  stripPrefix:
    prefixes:
      - /api/qwen25
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: strip-qwen3
spec:
  stripPrefix:
    prefixes:
      - /api/qwen3
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: vllm-ingress
spec:
  entryPoints:
    - web
  routes:
  - match: PathPrefix("/api/qwen25")
    kind: Rule
    services:
    - name: vllm-qwen25-05b
      port: 8001
    middlewares:
    - name: strip-qwen25
  - match: PathPrefix("/api/qwen3")
    kind: Rule
    services:
    - name: vllm-qwen3-06b
      port: 8002
    middlewares:
    - name: strip-qwen3
  - match: PathPrefix("/")
    kind: Rule
    services:
    - name: open-webui
      port: 8080

14.5 vLLM CPU 模式补丁（GPU 离线时备用） ​

bash
# 创建补丁脚本
cat << 'PATCHEOF' > /tmp/fix_cpu.pth
import importlib.metadata as _m; _m.version = (lambda _o: (lambda n, _f=_o, _c=_o("vllm"): (_c + "+cpu") if n == "vllm" else _f(n)))(_m.version)
PATCHEOF

# 通过 ConfigMap 注入容器
k3s kubectl create configmap vllm-cpu-patch \
  --from-file=fix_cpu.pth=/tmp/fix_cpu.pth -o yaml --dry-run=client \
  | k3s kubectl apply -f -

CPU 模式参数:

  • --device cpu --dtype float32 --swap-space 1 --max-num-seqs 4 --enforce-eager --disable-async-output-proc
  • VLLM_CPU_KVCACHE_SPACE=1

14.6 文档文件索引 ​

文件内容
deployment-guide.md本文档 - 完整部署指南
2080.md原始操作历史记录（含所有排错细节）
hami-vgpu.mdHAMi vGPU 实现原理详解
registry.mdDocker Registry 私有仓库部署
model.md模型文件路径说明
remote.mdSSH 连接信息

文档结束 -- 如有疑问，请参考 2080.md（完整操作历史）和 hami-vgpu.md（HAMi 原理详解）。