Kubernetes 部署 OpenStack 方案

版本: v1.0
日期: 2025-08
状态: 方案设计
关联: 本文档是 Kolla-Ansible 方案的替代路线，使用 Kubernetes 编排 OpenStack 服务

1. 方案概述

1.1 核心理念

将 OpenStack 各服务（Nova、Neutron、Keystone、Glance、Cinder 等）以 容器化 Pod 的形式运行在 Kubernetes 集群上，利用 K8s 的调度、自愈、滚动升级等能力管理 OpenStack 生命周期。

┌───────────────────────────────────────────────────────────────┐
│                    Kubernetes 集群                              │
│                                                               │
│  ┌──────────────────────────────────────────────────────┐     │
│  │              OpenStack Namespace (openstack)          │     │
│  │                                                       │     │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐             │     │
│  │  │ Keystone │ │ Nova API │ │ Neutron  │             │     │
│  │  │ (3 Pods) │ │ (5 Pods) │ │ (5 Pods) │             │     │
│  │  └──────────┘ └──────────┘ └──────────┘             │     │
│  │                                                       │     │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐             │     │
│  │  │ Glance   │ │ Cinder   │ │ Horizon  │             │     │
│  │  │ (3 Pods) │ │ (3 Pods) │ │ (3 Pods) │             │     │
│  │  └──────────┘ └──────────┘ └──────────┘             │     │
│  │                                                       │     │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐             │     │
│  │  │ Heat     │ │ Octavia  │ │ Barbican │             │     │
│  │  │ (3 Pods) │ │ (3 Pods) │ │ (3 Pods) │             │     │
│  │  └──────────┘ └──────────┘ └──────────┘             │     │
│  └──────────────────────────────────────────────────────┘     │
│                                                               │
│  ┌──────────────────────────────────────────────────────┐     │
│  │             Infra Namespace (infra)                   │     │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐             │     │
│  │  │ MariaDB  │ │ RabbitMQ │ │ Memcached│             │     │
│  │  │ (Galera) │ │ (Mirror) │ │(Cluster) │             │     │
│  │  └──────────┘ └──────────┘ └──────────┘             │     │
│  └──────────────────────────────────────────────────────┘     │
│                                                               │
│  ┌──────────────────────────────────────────────────────┐     │
│  │          Nova Compute (DaemonSet on Bare Metal)       │     │
│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐             │     │
│  │  │nova-cpu  │ │ libvirtd │ │ovn-ctrl  │ (x1000+)    │     │
│  │  └──────────┘ └──────────┘ └──────────┘             │     │
│  └──────────────────────────────────────────────────────┘     │
└───────────────────────────────────────────────────────────────┘

1.2 适用场景

场景	推荐方案	说明
传统虚拟化平台	Kolla-Ansible	简单直接，成熟稳定
云原生转型团队	K8s + OpenStack-Helm	统一 K8s 运维体系
同时运行 K8s 和 VM 工作负载	K8s + OpenStack-Helm	一套基础设施管理两种负载
大规模多集群管理	Airship / Atmosphere	企业级多站点管理

2. 方案对比与选型

2.1 三大方案对比

维度	OpenStack-Helm	Airship	Atmosphere
维护方	OpenStack 社区 (Vexxhost等)	AT&T 开源	Vexxhost
成熟度	⭐⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐
复杂度	中等	高	低 (封装好)
灵活性	高 (独立Helm Chart)	高 (YAML声明式)	中 (意见化方案)
K8s 依赖	需预先有 K8s 集群	自带 K8s 引导(Metal3)	使用 Cluster API
OpenStack 版本	2024.1/2024.2/2025.1	2024.x	2024.x
Ceph 集成	Rook-Ceph 或外部 Ceph	Rook-Ceph	Rook-Ceph
网络方案	OVN / OVS	OVN	OVN
学习曲线	中等	陡峭	平缓
社区活跃度	⭐⭐⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐

2.2 推荐选型: OpenStack-Helm

选择理由:

最成熟: 全球数千家企业生产验证，AT&T、Vexxhost、China Mobile 等大规模使用
最灵活: 每个 OpenStack 服务都是独立的 Helm Chart，可按需组合
生态完善: 与 Rook (Ceph)、OVN、Cert-Manager 等 K8s 生态无缝集成
渐进式采用: 可以先用 K8s 部署控制面，计算节点逐步迁移

推荐架构栈:
┌─────────────────────────────────────┐
│  OpenStack Services (Helm Charts)   │  ← OpenStack-Helm
├─────────────────────────────────────┤
│  OVN CNI + Multus CNI               │  ← 网络
├─────────────────────────────────────┤
│  Rook-Ceph / External Ceph          │  ← 存储
├─────────────────────────────────────┤
│  Kubernetes (kubeadm / Kubespray)   │  ← 容器编排
├─────────────────────────────────────┤
│  Bare Metal Nodes (1000+)           │  ← 物理服务器
└─────────────────────────────────────┘

3. 整体架构设计

3.1 架构分层

┌─────────────────────────────────────────────────────────────────┐
│                        用户访问层                                │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────────────┐  │
│  │ Ingress      │  │ OpenStack    │  │ Horizon Dashboard     │  │
│  │ Controller   │  │ CLI / SDK    │  │ (K8s Service)         │  │
│  └──────┬───────┘  └──────┬───────┘  └───────────┬───────────┘  │
│         │                 │                      │              │
│  ───────┼─────────────────┼──────────────────────┼───────────   │
│         │           API Gateway (VIP)            │              │
│  ───────┼─────────────────┼──────────────────────┼───────────   │
│         ▼                 ▼                      ▼              │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                   K8s Service Mesh                       │   │
│  │   (各 OpenStack 服务通过 K8s Service 相互发现)             │   │
│  └──────────┬──────────────┬──────────────────┬────────────┘   │
│             │              │                  │                  │
│  ┌──────────▼───┐ ┌───────▼──────┐ ┌────────▼────────┐        │
│  │ 控制面服务    │ │ 基础设施服务  │ │  数据面服务       │        │
│  │              │ │              │ │                  │        │
│  │ Keystone     │ │ MariaDB      │ │ Nova Compute     │        │
│  │ Nova API     │ │ Galera       │ │ (DaemonSet)      │        │
│  │ Nova Sched   │ │ RabbitMQ     │ │                  │        │
│  │ Neutron Srv  │ │ Memcached    │ │ ovn-controller   │        │
│  │ Glance API   │ │ Etcd         │ │ (DaemonSet)      │        │
│  │ Cinder API   │ │ Ingress      │ │                  │        │
│  │ Heat API     │ │ Cert-Manager │ │ libvirtd         │        │
│  │ Octavia API  │ │              │ │ (DaemonSet)      │        │
│  │ Barbican     │ │              │ │                  │        │
│  └──────────────┘ └──────────────┘ └──────────────────┘        │
└─────────────────────────────────────────────────────────────────┘

3.2 Namespace 规划

Namespace	用途	部署内容
`openstack`	OpenStack 服务	Keystone, Nova, Neutron, Glance, Cinder, Heat, Octavia, Horizon
`infra`	基础设施	MariaDB Galera, RabbitMQ, Memcached, Etcd
`network`	网络组件	OVN NB/SB DB, ovn-northd, Ingress Controller
`storage`	存储	Rook-Ceph Operator (若使用 Rook)
`monitoring`	监控	Prometheus, Grafana, Loki, AlertManager
`compute`	计算面	Nova Compute DaemonSet, libvirtd, ovn-controller

3.3 节点角色设计

角色	数量	K8s 角色	部署服务
K8s Controller	5	master (control-plane)	K8s API, etcd, scheduler, OVN NB/SB
K8s Worker (Infra)	6	worker	MariaDB, RabbitMQ, Memcached, 监控
K8s Worker (API)	5	worker	OpenStack API 服务
Bare Metal Compute	1000+	worker (特殊标签)	Nova Compute, libvirtd, ovn-controller

节点标签设计:
  k8s-controller:  openstack-control-plane=enabled
  infra-worker:    openstack-control-plane=enabled, openstack-infra=enabled
  api-worker:      openstack-control-plane=enabled, openstack-api=enabled
  compute-node:    openstack-compute-node=enabled

4. Kubernetes 集群规划

4.1 K8s 集群部署 (Kubespray)

bash

# 1. 下载 Kubespray
git clone -b v2.26.0 https://github.com/kubernetes-sigs/kubespray.git
cd kubespray
pip3 install -r requirements.txt

# 2. 生成 inventory
cp -rfp inventory/sample inventory/openstack-k8s

# 3. 配置 inventory/openstack-k8s/hosts.yaml

hosts.yaml 配置:

yaml

all:
  hosts:
    # K8s Control Plane (5 节点高可用)
    k8s-ctrl-01:
      ansible_host: 10.0.1.11
      ip: 10.0.1.11
      access_ip: 10.0.1.11
    k8s-ctrl-02:
      ansible_host: 10.0.1.12
      ip: 10.0.1.12
    k8s-ctrl-03:
      ansible_host: 10.0.1.13
      ip: 10.0.1.13
    k8s-ctrl-04:
      ansible_host: 10.0.1.14
      ip: 10.0.1.14
    k8s-ctrl-05:
      ansible_host: 10.0.1.15
      ip: 10.0.1.15

    # Infra Worker (运行 MariaDB, RabbitMQ 等)
    infra-01:
      ansible_host: 10.0.1.21
      ip: 10.0.1.21
    infra-02:
      ansible_host: 10.0.1.22
      ip: 10.0.1.22
    infra-03:
      ansible_host: 10.0.1.23
      ip: 10.0.1.23
    infra-04:
      ansible_host: 10.0.1.24
      ip: 10.0.1.24
    infra-05:
      ansible_host: 10.0.1.25
      ip: 10.0.1.25
    infra-06:
      ansible_host: 10.0.1.26
      ip: 10.0.1.26

    # API Worker (运行 OpenStack API 服务)
    api-01:
      ansible_host: 10.0.1.31
      ip: 10.0.1.31
    api-02:
      ansible_host: 10.0.1.32
      ip: 10.0.1.32
    api-03:
      ansible_host: 10.0.1.33
      ip: 10.0.1.33
    api-04:
      ansible_host: 10.0.1.34
      ip: 10.0.1.34
    api-05:
      ansible_host: 10.0.1.35
      ip: 10.0.1.35

    # Compute Nodes (通过脚本批量生成)
    # compute-001 ~ compute-1000

  children:
    kube_control_plane:
      hosts:
        k8s-ctrl-01:
        k8s-ctrl-02:
        k8s-ctrl-03:
        k8s-ctrl-04:
        k8s-ctrl-05:
    kube_node:
      hosts:
        infra-01:
        infra-02:
        infra-03:
        infra-04:
        infra-05:
        infra-06:
        api-01:
        api-02:
        api-03:
        api-04:
        api-05:
        # compute-001 ~ compute-1000
    etcd:
      hosts:
        k8s-ctrl-01:
        k8s-ctrl-02:
        k8s-ctrl-03:
        k8s-ctrl-04:
        k8s-ctrl-05:

4.2 K8s 集群核心配置

yaml

# inventory/openstack-k8s/group_vars/k8s_cluster/k8s-cluster.yml

# K8s 版本
kube_version: v1.31.0

# 容器运行时
container_manager: containerd

# 网络插件 - 使用 Calico (与 OVN 配合)
kube_network_plugin: calico
calico_backend: bird

# Service CIDR
kube_service_addresses: 10.233.0.0/18

# Pod CIDR
kube_pods_subnet: 10.233.64.0/18

# 高可用配置
kube_apiserver_count: 5
loadbalancer_apiserver_localhost: true

# 大规模集群优化
kubelet_max_pods: 110
kube_proxy_mode: ipvs

# etcd 优化 (大规模)
etcd_extra_args:
  heartbeat-interval: "200"
  election-timeout: "2000"
  max-request-bytes: "33554432"
  quota-backend-bytes: "8589934592"

# Feature Gates
kube_feature_gates:
  - "GracefulNodeShutdown=true"
  - "TopologyAwareHints=true"

4.3 部署 K8s 集群

bash

# 1. 基础配置 (SSH、NTP、内核参数)
ansible-playbook -i inventory/openstack-k8s/hosts.yaml \
  -b cluster.yml --tags bootstrap

# 2. 完整部署
ansible-playbook -i inventory/openstack-k8s/hosts.yaml \
  -b cluster.yml -vvv

# 3. 验证集群
kubectl get nodes -o wide
kubectl get cs

# 4. 给节点打标签
# Infra 节点
for node in infra-{01..06}; do
  kubectl label node $node openstack-control-plane=enabled
  kubectl label node $node openstack-infra=enabled
done

# API 节点
for node in api-{01..05}; do
  kubectl label node $node openstack-control-plane=enabled
  kubectl label node $node openstack-api=enabled
done

# 计算节点 (批量)
for i in $(seq -w 1 1000); do
  kubectl label node compute-$i openstack-compute-node=enabled
done

4.4 大规模 K8s 优化

yaml

# K8s API Server 优化 (kube-apiserver 额外参数)
kube_apiserver_extra_args:
  max-requests-inflight: "3000"
  max-mutating-requests-inflight: "1000"
  event-ttl: "1h"
  audit-log-maxage: "30"

# Controller Manager 优化
kube_controller_manager_extra_args:
  node-cidr-mask-size: "24"
  node-monitor-grace-period: "40s"
  pod-eviction-timeout: "5m0s"

# Scheduler 优化
kube_scheduler_extra_args:
  percentage-of-nodes-to-score: "30"

5. OpenStack-Helm 部署详解

5.1 安装 Helm 和依赖

bash

# 1. 安装 Helm 3
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm version

# 2. 添加 OpenStack-Helm 仓库
helm repo add openstack-helm \
  https://tarballs.opendev.org/openstack/openstack-helm/charts/
helm repo update

# 3. 添加依赖 Helm 仓库
helm repo add jetstack https://charts.jetstack.io
helm repo add rook-release https://charts.rook.io/release
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

# 4. 克隆 OpenStack-Helm 源码 (用于自定义 values)
git clone https://opendev.org/openstack/openstack-helm.git
cd openstack-helm

# 5. 克隆 OpenStack-Helm-Infra
git clone https://opendev.org/openstack/openstack-helm-infra.git
cd openstack-helm-infra

5.2 部署顺序

部署流程:
  1. 基础设施层 → 2. 网络层 → 3. 存储层 → 4. OpenStack 服务

详细顺序:
  Step 1:  cert-manager, ingress-nginx
  Step 2:  MariaDB (Galera), RabbitMQ, Memcached, Etcd
  Step 3:  OVN (NB/SB DB + northd + controller)
  Step 4:  Rook-Ceph (或配置外部 Ceph)
  Step 5:  Keystone (身份认证)
  Step 6:  Glance (镜像)
  Step 7:  Nova (计算)
  Step 8:  Neutron (网络)
  Step 9:  Cinder (块存储)
  Step 10: Heat (编排)
  Step 11: Octavia (负载均衡)
  Step 12: Horizon (Dashboard)
  Step 13: Barbican (密钥管理)

5.3 自动化部署脚本

bash

#!/bin/bash
# deploy-openstack-helm.sh
# OpenStack-Helm 一键部署脚本

set -euo pipefail

NAMESPACE_OS="openstack"
NAMESPACE_INFRA="infra"
NAMESPACE_NET="network"
VALUES_DIR="./values"

# 创建 Namespace
kubectl create namespace ${NAMESPACE_OS} || true
kubectl create namespace ${NAMESPACE_INFRA} || true
kubectl create namespace ${NAMESPACE_NET} || true

# ==========================================
# Step 1: Cert-Manager
# ==========================================
echo ">>> [1/12] 部署 Cert-Manager..."
helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set installCRDs=true \
  --wait --timeout 5m

# ==========================================
# Step 2: Ingress Controller
# ==========================================
echo ">>> [2/12] 部署 Ingress Nginx..."
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace \
  -f ${VALUES_DIR}/ingress-nginx.yaml \
  --wait --timeout 5m

# ==========================================
# Step 3: MariaDB Galera Cluster
# ==========================================
echo ">>> [3/12] 部署 MariaDB Galera..."
helm upgrade --install mariadb \
  ./openstack-helm-infra/charts/mariadb \
  --namespace ${NAMESPACE_INFRA} \
  -f ${VALUES_DIR}/mariadb.yaml \
  --wait --timeout 10m

# ==========================================
# Step 4: RabbitMQ
# ==========================================
echo ">>> [4/12] 部署 RabbitMQ..."
helm upgrade --install rabbitmq \
  ./openstack-helm-infra/charts/rabbitmq \
  --namespace ${NAMESPACE_INFRA} \
  -f ${VALUES_DIR}/rabbitmq.yaml \
  --wait --timeout 10m

# ==========================================
# Step 5: Memcached
# ==========================================
echo ">>> [5/12] 部署 Memcached..."
helm upgrade --install memcached \
  ./openstack-helm-infra/charts/memcached \
  --namespace ${NAMESPACE_INFRA} \
  -f ${VALUES_DIR}/memcached.yaml \
  --wait --timeout 5m

# ==========================================
# Step 6: OVN
# ==========================================
echo ">>> [6/12] 部署 OVN..."
helm upgrade --install ovn \
  ./openstack-helm/charts/ovn \
  --namespace ${NAMESPACE_NET} \
  -f ${VALUES_DIR}/ovn.yaml \
  --wait --timeout 10m

# ==========================================
# Step 7: Keystone
# ==========================================
echo ">>> [7/12] 部署 Keystone..."
helm upgrade --install keystone \
  ./openstack-helm/charts/keystone \
  --namespace ${NAMESPACE_OS} \
  -f ${VALUES_DIR}/keystone.yaml \
  --wait --timeout 10m

# ==========================================
# Step 8: Glance
# ==========================================
echo ">>> [8/12] 部署 Glance..."
helm upgrade --install glance \
  ./openstack-helm/charts/glance \
  --namespace ${NAMESPACE_OS} \
  -f ${VALUES_DIR}/glance.yaml \
  --wait --timeout 10m

# ==========================================
# Step 9: Nova
# ==========================================
echo ">>> [9/12] 部署 Nova..."
helm upgrade --install nova \
  ./openstack-helm/charts/nova \
  --namespace ${NAMESPACE_OS} \
  -f ${VALUES_DIR}/nova.yaml \
  --wait --timeout 15m

# ==========================================
# Step 10: Neutron
# ==========================================
echo ">>> [10/12] 部署 Neutron..."
helm upgrade --install neutron \
  ./openstack-helm/charts/neutron \
  --namespace ${NAMESPACE_OS} \
  -f ${VALUES_DIR}/neutron.yaml \
  --wait --timeout 10m

# ==========================================
# Step 11: Cinder
# ==========================================
echo ">>> [11/12] 部署 Cinder..."
helm upgrade --install cinder \
  ./openstack-helm/charts/cinder \
  --namespace ${NAMESPACE_OS} \
  -f ${VALUES_DIR}/cinder.yaml \
  --wait --timeout 10m

# ==========================================
# Step 12: Horizon
# ==========================================
echo ">>> [12/12] 部署 Horizon..."
helm upgrade --install horizon \
  ./openstack-helm/charts/horizon \
  --namespace ${NAMESPACE_OS} \
  -f ${VALUES_DIR}/horizon.yaml \
  --wait --timeout 5m

echo ""
echo "=== OpenStack-Helm 部署完成 ==="
echo "Dashboard: https://openstack.example.com"
echo "CLI: source openstack-rc.sh"

6. 基础设施服务部署

6.1 MariaDB Galera Cluster

yaml

# values/mariadb.yaml
---
release_group: mariadb

# 5 节点 Galera (奇数, 容忍 2 节点故障)
replicas: 5

# Pod 分布
pod:
  replicas:
    server: 5
  affinity:
    antiAffinity:
      topologyKey: kubernetes.io/hostname

# 镜像
images:
  tags:
    mariadb: docker.io/library/mariadb:10.11

# 存储
conf:
  database:
    root_user: root
    admin_user: openstack

# 资源限制
resources:
  server:
    requests:
      memory: "8Gi"
      cpu: "4"
    limits:
      memory: "16Gi"
      cpu: "8"

# 持久卷
storage:
  server:
    class: "local-storage"
    size: "200Gi"

# MariaDB 优化参数
conf:
  mysql:
    innodb_buffer_pool_size: "12G"
    innodb_log_file_size: "2G"
    innodb_flush_log_at_trx_commit: "1"
    innodb_flush_method: "O_DIRECT"
    max_connections: "4096"
    thread_cache_size: "128"
    table_open_cache: "4096"
    wsrep_slave_threads: "8"
    wsrep_sync_wait: "1"

# 每个 OpenStack 服务数据库
databases:
  - keystone
  - nova
  - nova_api
  - nova_cell0
  - placement
  - neutron
  - glance
  - cinder
  - heat
  - octavia
  - barbican
  - designate

6.2 RabbitMQ Cluster

yaml

# values/rabbitmq.yaml
---
release_group: rabbitmq

# 5 节点镜像队列集群
replicas: 5

images:
  tags:
    rabbitmq: docker.io/library/rabbitmq:3.13-management

# 集群配置
conf:
  rabbitmq:
    cluster_formation:
      peer_discovery_backend: rabbit_peer_discovery_k8s
      k8s_host: kubernetes.default
      k8s_address_type: hostname
    queue_master_locator: min-masters
    # 大规模优化
    channel_max: 65535
    heartbeat: 60
    collect_statistics_interval: 30000

pod:
  affinity:
    antiAffinity:
      topologyKey: kubernetes.io/hostname
  resources:
    rabbitmq:
      requests:
        memory: "4Gi"
        cpu: "2"
      limits:
        memory: "8Gi"
        cpu: "4"

storage:
  class: "local-storage"
  size: "50Gi"

6.3 Memcached Cluster

yaml

# values/memcached.yaml
---
replicas: 3

images:
  tags:
    memcached: docker.io/library/memcached:1.6

pod:
  resources:
    memcached:
      requests:
        memory: "4Gi"
        cpu: "2"
      limits:
        memory: "8Gi"
        cpu: "4"

conf:
  memcached:
    max_memory: 6144     # 6GB
    max_connections: 8192
    threads: 8

7. OpenStack 核心服务部署

7.1 Keystone

yaml

# values/keystone.yaml
---
release_group: keystone

images:
  tags:
    keystone: docker.io/openstackhelm/keystone:2024.2-ubuntu_jammy
    bootstrap: docker.io/openstackhelm/keystone:2024.2-ubuntu_jammy

pod:
  replicas:
    api: 5
    bootstrap: 1

network:
  api:
    ingress:
      classes:
        namespace: "nginx"
        cluster: "nginx"
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /

conf:
  keystone:
    DEFAULT:
      debug: false
      max_token_size: 255
    token:
      provider: fernet
      expiration: 3600
    fernet_tokens:
      max_active_keys: 7
    cache:
      enabled: true
      backend: dogpile.cache.pymemcache
      memcache_servers: memcached.infra.svc.cluster.local:11211
    database:
      connection: >-
        mysql+pymysql://keystone:KEYSTONE_DB_PASS@mariadb.infra.svc.cluster.local:3306/keystone
      max_pool_size: 20
      max_overflow: 30
    oslo_messaging_rabbit:
      rabbit_host: rabbitmq.infra.svc.cluster.local
      rabbit_userid: openstack
      rabbit_password: xxx

resources:
  api:
    requests:
      memory: "1Gi"
      cpu: "1"
    limits:
      memory: "2Gi"
      cpu: "2"

bootstrap:
  enabled: true
  script:
    admin_project_name: admin
    admin_user_name: admin

7.2 Nova (控制面 + 计算面分离)

Nova 控制面 (API, Conductor, Scheduler):

yaml

# values/nova.yaml
---
release_group: nova

images:
  tags:
    nova_api: docker.io/openstackhelm/nova:2024.2-ubuntu_jammy
    nova_conductor: docker.io/openstackhelm/nova:2024.2-ubuntu_jammy
    nova_scheduler: docker.io/openstackhelm/nova:2024.2-ubuntu_jammy
    nova_compute: docker.io/openstackhelm/nova:2024.2-ubuntu_jammy
    nova_novncproxy: docker.io/openstackhelm/nova:2024.2-ubuntu_jammy
    bootstrap: docker.io/openstackhelm/nova:2024.2-ubuntu_jammy

pod:
  replicas:
    api_metadata: 3
    osapi: 5          # Nova API 5 副本
    conductor: 5      # Nova Conductor 5 副本
    scheduler: 3
    novncproxy: 3

  nodeSelector:
    openstack-api: enabled

conf:
  nova:
    DEFAULT:
      debug: false
      osapi_compute_workers: 16
      metadata_workers: 8
      use_neutron: true
      firewall_driver: nova.virt.firewall.NoopFirewallDriver
      network_api_class: nova.network.neutronv2.api.API
    scheduler:
      workers: 8
      driver: filter_scheduler
    filter_scheduler:
      enabled_filters: >-
        AvailabilityZoneFilter,ComputeFilter,
        ComputeCapabilitiesFilter,ImagePropertiesFilter,
        ServerGroupAffinityFilter,ServerGroupAntiAffinityFilter,
        DiskFilter,RamFilter,NUMATopologyFilter
      max_placement_results: 1000
    conductor:
      workers: 16
    database:
      connection: >-
        mysql+pymysql://nova:NOVA_DB_PASS@mariadb.infra.svc.cluster.local:3306/nova
      max_pool_size: 30
      max_overflow: 50
    oslo_messaging_rabbit:
      rabbit_host: rabbitmq.infra.svc.cluster.local
      rabbit_userid: openstack
      rabbit_password: xxx
      rabbit_ha_queues: true
      heartbeat_timeout_threshold: 60
    cache:
      enabled: true
      backend: dogpile.cache.pymemcache
      memcache_servers: memcached.infra.svc.cluster.local:11211
    libvirt:
      images_type: rbd
      images_rbd_pool: vms
      images_rbd_ceph_conf: /etc/ceph/ceph.conf
      rbd_user: nova
      live_migration_uri: qemu+tcp://%s/system
      live_migration_permit_auto_converge: true
      live_migration_permit_post_copy: true

resources:
  osapi:
    requests:
      memory: "2Gi"
      cpu: "2"
    limits:
      memory: "4Gi"
      cpu: "4"
  conductor:
    requests:
      memory: "2Gi"
      cpu: "2"
    limits:
      memory: "8Gi"
      cpu: "4"

Nova 计算面 (DaemonSet, 部署在每台裸金属计算节点):

yaml

# values/nova-compute.yaml (追加到 nova.yaml 中)
---
# Nova Compute 以 DaemonSet 方式运行在每台计算节点

pod:
  replicas:
    compute: 1   # DaemonSet 每节点一个

  # 只在计算节点上调度
  nodeSelector:
    openstack-compute-node: enabled

  # 允许在 tainted 节点上运行
  tolerations:
    - key: "openstack-compute"
      operator: "Exists"
      effect: "NoSchedule"

conf:
  nova:
    DEFAULT:
      compute_driver: libvirt.LibvirtDriver
      resume_guests_state_on_host_boot: true
      update_resources_interval: 60
    libvirt:
      virt_type: kvm
      cpu_mode: host-passthrough
      images_type: rbd
      images_rbd_pool: vms
      images_rbd_ceph_conf: /etc/ceph/ceph.conf
      rbd_user: nova
      rbd_secret_uuid: CEPH_NOVA_SECRET_UUID
      disk_cachemodes: network=writeback
      hw_disk_discard: unmap
      live_migration_uri: qemu+tcp://%s/system
      live_migration_tunnelled: false
      live_migration_permit_auto_converge: true
      live_migration_permit_post_copy: true
      num_pcie_ports: 28
    vnc:
      enabled: true
      server_listen: 0.0.0.0
      server_proxyclient_address: "${my_ip}"

# 计算节点需要特权 (访问 KVM /dev/kvm)
security_context:
  nova:
    compute:
      privileged: true
      runAsUser: 0

# 挂载宿主机设备和 Ceph 配置
volumes:
  - name: dev-kvm
    hostPath:
      path: /dev/kvm
  - name: ceph-conf
    hostPath:
      path: /etc/ceph
  - name: libvirt-socket
    hostPath:
      path: /var/run/libvirt/libvirt-sock
  - name: nova-instances
    hostPath:
      path: /var/lib/nova/instances

7.3 Neutron (配合 OVN)

yaml

# values/neutron.yaml
---
release_group: neutron

images:
  tags:
    neutron_server: docker.io/openstackhelm/neutron:2024.2-ubuntu_jammy
    bootstrap: docker.io/openstackhelm/neutron:2024.2-ubuntu_jammy

pod:
  replicas:
    server: 5       # Neutron Server 5 副本
  nodeSelector:
    openstack-api: enabled

conf:
  neutron:
    DEFAULT:
      debug: false
      core_plugin: ml2
      service_plugins: router, segments, trunk
      api_workers: 16
      rpc_workers: 8
      allow_overlapping_ips: true
      router_distributed: true
    database:
      connection: >-
        mysql+pymysql://neutron:NEUTRON_DB_PASS@mariadb.infra.svc.cluster.local:3306/neutron
      max_pool_size: 30
    oslo_messaging_rabbit:
      rabbit_host: rabbitmq.infra.svc.cluster.local
      rabbit_userid: openstack
      rabbit_password: xxx
    cache:
      enabled: true
      backend: dogpile.cache.pymemcache
      memcache_servers: memcached.infra.svc.cluster.local:11211

  plugins:
    ml2_conf:
      ml2:
        type_drivers: geneve,vlan,flat
        tenant_network_types: geneve
        mechanism_drivers: ovn
        extension_drivers: port_security,qos
      ml2_type_geneve:
        vni_ranges: 1:65536
        max_header_size: 38
      ml2_type_vlan:
        network_vlan_ranges: physnet1:1:4094
      ovn:
        ovn_nb_connection: tcp:ovn-nb.network.svc.cluster.local:6641
        ovn_sb_connection: tcp:ovn-sb.network.svc.cluster.local:6642
        ovn_l3_scheduler: leastloaded
        ovn_metadata_enabled: true
        dns_servers: 10.0.0.53

resources:
  server:
    requests:
      memory: "2Gi"
      cpu: "2"
    limits:
      memory: "8Gi"
      cpu: "8"

7.4 Glance (镜像服务)

yaml

# values/glance.yaml
---
release_group: glance

images:
  tags:
    glance_api: docker.io/openstackhelm/glance:2024.2-ubuntu_jammy

pod:
  replicas:
    api: 3

conf:
  glance:
    DEFAULT:
      debug: false
      workers: 8
    glance_store:
      stores: rbd
      default_store: rbd
      rbd_store_pool: images
      rbd_store_user: glance
      rbd_store_cephconf: /etc/ceph/ceph.conf
      rbd_store_chunk_size: 8
    database:
      connection: >-
        mysql+pymysql://glance:GLANCE_DB_PASS@mariadb.infra.svc.cluster.local:3306/glance
    cache:
      enabled: true
      image_cache_dir: /var/lib/glance/image-cache

resources:
  api:
    requests:
      memory: "2Gi"
      cpu: "2"
    limits:
      memory: "4Gi"
      cpu: "4"

7.5 Cinder (块存储)

yaml

# values/cinder.yaml
---
release_group: cinder

images:
  tags:
    cinder_api: docker.io/openstackhelm/cinder:2024.2-ubuntu_jammy
    cinder_scheduler: docker.io/openstackhelm/cinder:2024.2-ubuntu_jammy
    cinder_volume: docker.io/openstackhelm/cinder:2024.2-ubuntu_jammy

pod:
  replicas:
    api: 5
    scheduler: 3
    volume: 3

conf:
  cinder:
    DEFAULT:
      debug: false
      enabled_backends: ceph-rbd
      osapi_volume_workers: 8
    database:
      connection: >-
        mysql+pymysql://cinder:CINDER_DB_PASS@mariadb.infra.svc.cluster.local:3306/cinder
    ceph-rbd:
      volume_driver: cinder.volume.drivers.rbd.RBDDriver
      rbd_pool: volumes
      rbd_ceph_conf: /etc/ceph/ceph.conf
      rbd_user: cinder
      rbd_secret_uuid: CEPH_CINDER_SECRET_UUID
      rbd_store_chunk_size: 4
      rbd_max_clone_depth: 5
      report_discard_supported: true
      volume_backend_name: ceph-rbd

resources:
  api:
    requests:
      memory: "1Gi"
      cpu: "1"
    limits:
      memory: "4Gi"
      cpu: "4"
  volume:
    requests:
      memory: "1Gi"
      cpu: "1"
    limits:
      memory: "4Gi"
      cpu: "4"

8. 网络集成 (OVN)

8.1 OVN 在 K8s 上的部署

yaml

# values/ovn.yaml
---
release_group: ovn

images:
  tags:
    ovn_northd: docker.io/openstackhelm/ovn:latest
    ovn_nb_db: docker.io/openstackhelm/ovn:latest
    ovn_sb_db: docker.io/openstackhelm/ovn:latest
    ovn_controller: docker.io/openstackhelm/ovn:latest

# OVN Northbound DB (Raft 集群)
labels:
  nb_db:
    node_selector_key: openstack-control-plane
    node_selector_value: enabled
  sb_db:
    node_selector_key: openstack-control-plane
    node_selector_value: enabled

# NB/SB DB 各 3 节点 (Raft 一致性)
pod:
  replicas:
    nb_db: 3
    sb_db: 3
    northd: 3

  # OVN Controller 以 DaemonSet 运行在所有计算节点
  daemonset:
    ovn_controller:
      nodeSelector:
        openstack-compute-node: enabled

# OVN 网络参数
conf:
  ovn:
    nb_db:
      listen_port: 6641
    sb_db:
      listen_port: 6642
    northd:
      n_threads: 8
      probe_interval: 15000
    controller:
      encap_type: geneve
      encap_csum: true
      bridge_mappings: "physnet1:br-ex"
      openflow_probe_interval: 60
      inactivity_probe: 60000

# 需要特权模式 (操作 OVS 内核模块)
security_context:
  ovn_controller:
    privileged: true

8.2 多网络平面 (Multus CNI)

bash

# 安装 Multus CNI (允许 Pod 多网络接口)
kubectl apply -f \
  https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/master/deployments/multus-daemonset-thick.yml

yaml

# 创建 OpenStack 网络附加定义
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: openstack-management
  namespace: openstack
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "name": "openstack-mgmt",
      "type": "macvlan",
      "master": "bond0.100",
      "mode": "bridge",
      "ipam": {
        "type": "static",
        "addresses": [
          { "address": "10.0.1.100/24" }
        ]
      }
    }
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: openstack-storage
  namespace: openstack
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "name": "openstack-storage",
      "type": "macvlan",
      "master": "bond0.200",
      "mode": "bridge",
      "ipam": {
        "type": "dhcp"
      }
    }

9. 存储集成 (Ceph)

9.1 方案一: Rook-Ceph (K8s 原生管理)

bash

# 安装 Rook Operator
helm repo add rook-release https://charts.rook.io/release
helm install rook-ceph rook-release/rook-ceph \
  --namespace rook-ceph --create-namespace \
  --set csi.cephFSPlugin.enabled=true \
  --set csi.rbdPlugin.enabled=true

# 安装 Rook-Ceph Cluster
helm install rook-ceph-cluster rook-release/rook-ceph-cluster \
  --namespace rook-ceph \
  -f values/rook-ceph-cluster.yaml

yaml

# values/rook-ceph-cluster.yaml
---
operatorNamespace: rook-ceph

spec:
  cephVersion:
    image: quay.io/ceph/ceph:v19.2

  dataDirHostPath: /var/lib/rook

  # Mon 3 节点
  mon:
    count: 3
    allowMultiplePerNode: false

  # Mgr 2 节点
  mgr:
    count: 2
    modules:
      - name: dashboard
        enabled: true
      - name: prometheus
        enabled: true

  # 存储配置 - 使用裸磁盘
  storage:
    useAllNodes: false
    useAllDevices: false
    nodes:
      - name: "ceph-osd-01"
        devices:
          - name: "/dev/sda"
          - name: "/dev/sdb"
          # ... 共 12 块 HDD
        config:
          databasePathDevice: "/dev/nvme0n1"
          walPathDevice: "/dev/nvme0n1"
          osdsPerDevice: "1"
      # ... 其余 29 个 OSD 节点

  resources:
    osd:
      requests:
        cpu: "2"
        memory: "8Gi"
      limits:
        memory: "12Gi"
    mon:
      requests:
        cpu: "1"
        memory: "2Gi"

9.2 方案二: 外部 Ceph 集群集成

如果 Ceph 独立部署（如当前方案），通过 Secret 和 ConfigMap 集成:

bash

# 1. 创建 Ceph 配置文件 ConfigMap
kubectl create configmap ceph-conf \
  --namespace openstack \
  --from-file=ceph.conf=/etc/ceph/ceph.conf

# 2. 创建 Ceph 密钥 Secret
# Nova 用户
ceph auth get-key client.nova | base64 > nova-ceph-key.b64
kubectl create secret generic ceph-nova-secret \
  --namespace openstack \
  --from-file=key=nova-ceph-key.b64

# Glance 用户
ceph auth get-key client.glance | base64 > glance-ceph-key.b64
kubectl create secret generic ceph-glance-secret \
  --namespace openstack \
  --from-file=key=glance-ceph-key.b64

# Cinder 用户
ceph auth get-key client.cinder | base64 > cinder-ceph-key.b64
kubectl create secret generic ceph-cinder-secret \
  --namespace openstack \
  --from-file=key=cinder-ceph-key.b64

9.3 Ceph RBD StorageClass

yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-rbd
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: rook-ceph.rbd.csi.ceph.com
parameters:
  clusterID: rook-ceph
  pool: volumes
  imageFormat: "2"
  imageFeatures: layering
  csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
  csi.storage.k8s.io/provisioner-secret-namespace: rook-ceph
  csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
  csi.storage.k8s.io/node-stage-secret-namespace: rook-ceph
reclaimPolicy: Delete
allowVolumeExpansion: true

10. 高可用与故障恢复

10.1 K8s 层面的 HA

组件	HA 策略	副本数
K8s API Server	多 master + etcd	5
etcd	Raft 共识	5
MariaDB	Galera 同步复制	5
RabbitMQ	镜像队列	5
OpenStack API	K8s Deployment + HPA	3-5
OVN NB/SB DB	Raft	3
ovn-controller	DaemonSet (每节点)	N/A
Nova Compute	DaemonSet (每计算节点)	N/A

10.2 Pod 反亲和性 (跨节点/跨机架)

yaml

# 确保 OpenStack 服务跨机架分布
podAntiAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
          - key: application
            operator: In
            values:
              - keystone
      topologyKey: topology.kubernetes.io/zone
  preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        labelSelector:
          matchExpressions:
            - key: application
              operator: In
              values:
                - keystone
        topologyKey: kubernetes.io/hostname

10.3 PodDisruptionBudget

yaml

# 确保滚动升级时保持服务可用
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: keystone-pdb
  namespace: openstack
spec:
  minAvailable: 2
  selector:
    matchLabels:
      application: keystone
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: nova-api-pdb
  namespace: openstack
spec:
  minAvailable: 3
  selector:
    matchLabels:
      application: nova
      component: osapi

10.4 自动扩缩容 (HPA)

yaml

# HPA - 根据负载自动扩缩 Nova API
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nova-api-hpa
  namespace: openstack
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nova-osapi
  minReplicas: 5
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

11. 运维管理

11.1 监控 (Prometheus + Grafana)

bash

# 安装 kube-prometheus-stack
helm repo add prometheus-community \
  https://prometheus-community.github.io/helm-charts

helm install kube-prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring --create-namespace \
  -f values/kube-prometheus.yaml

yaml

# values/kube-prometheus.yaml
---
prometheus:
  prometheusSpec:
    retention: 30d
    storageSpec:
      volumeClaimTemplate:
        spec:
          storageClassName: ceph-rbd
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 500Gi
    additionalScrapeConfigs:
      - job_name: 'openstack-keystone'
        kubernetes_sd_configs:
          - role: endpoints
            namespaces:
              names: ['openstack']
        relabel_configs:
          - source_labels: [__meta_kubernetes_service_name]
            regex: keystone-api
            action: keep

grafana:
  adminPassword: xxx
  dashboardProviders:
    dashboardproviders.yaml:
      apiVersion: 1
      providers:
        - name: openstack
          orgId: 1
          folder: OpenStack
          type: file
          options:
            path: /var/lib/grafana/dashboards/openstack
  dashboards:
    openstack:
      nova-overview:
        gnetId: 14218
        datasource: Prometheus
      ceph-cluster:
        gnetId: 2842
        datasource: Prometheus

11.2 日志收集 (Loki)

bash

# 安装 Loki + Promtail
helm install loki grafana/loki-stack \
  --namespace monitoring \
  --set promtail.enabled=true \
  --set loki.persistence.enabled=true \
  --set loki.persistence.size=200Gi

11.3 常用运维命令

bash

# 查看所有 OpenStack 服务 Pod 状态
kubectl get pods -n openstack -o wide

# 查看某个服务的日志
kubectl logs -n openstack deployment/keystone-api -f --tail=100

# 滚动升级 Keystone
helm upgrade keystone ./openstack-helm/charts/keystone \
  --namespace openstack \
  -f values/keystone.yaml \
  --set images.tags.keystone=docker.io/openstackhelm/keystone:2025.1-ubuntu_jammy

# 回滚 Keystone
helm rollback keystone 1 --namespace openstack

# 扩容 Nova API
kubectl scale deployment nova-osapi -n openstack --replicas=8

# 查看计算节点状态
kubectl get daemonset -n openstack nova-compute
kubectl get pods -n openstack -l application=nova,component=compute

# 添加新计算节点:
# 1. 在新节点安装 K8s kubelet
kubespray scale --limit compute-1001 -i inventory/hosts.yaml

# 2. 打标签
kubectl label node compute-1001 openstack-compute-node=enabled

# 3. Nova Compute DaemonSet 自动调度到新节点

# 4. 发现新 compute host
kubectl exec -n openstack deployment/nova-conductor -- \
  nova-manage cell_v2 discover_hosts

11.4 升级策略

bash

# OpenStack 服务升级流程:
# 1. 更新 Helm Chart values 中的镜像 tag
# 2. 执行 helm upgrade (自动滚动更新)
# 3. 验证服务状态

# 控制面升级
helm upgrade nova ./openstack-helm/charts/nova \
  --namespace openstack \
  -f values/nova.yaml \
  --set images.tags.nova_api=docker.io/openstackhelm/nova:2025.1-ubuntu_jammy

# 计算节点分批升级 (每批 100 节点)
for batch in $(seq 1 10); do
  echo "=== Upgrading compute batch $batch ==="
  for i in $(seq $((batch*100-99)) $((batch*100))); do
    kubectl cordon compute-$(printf "%03d" $i)
  done
  kubectl rollout restart daemonset/nova-compute -n openstack
  for i in $(seq $((batch*100-99)) $((batch*100))); do
    kubectl uncordon compute-$(printf "%03d" $i)
  done
  sleep 120
done

12. 与 Kolla-Ansible 对比

维度	Kolla-Ansible (当前方案)	K8s + OpenStack-Helm
部署方式	Ansible + Docker	Helm charts + K8s
容器运行时	Docker (每节点独立)	containerd (K8s管理)
服务管理	Docker/systemd	K8s Deployment/DaemonSet
配置管理	globals.yml + Jinja2	Helm values.yaml
高可用	HAProxy + Keepalived	K8s Service + Ingress
滚动升级	Ansible serial 批次	K8s Rolling Update
回滚	手动切换 Docker tag	helm rollback (一键)
自愈能力	无 (依赖外部监控)	K8s 自动重启失败Pod
扩缩容	手动修改 inventory	kubectl scale / HPA
日志管理	Docker logs + Filebeat	K8s stdout + Promtail
监控	Prometheus + exporter	kube-prometheus-stack
学习成本	Ansible (中)	K8s + Helm (高)
运维复杂度	低 (简单直观)	高 (K8s概念多)
社区成熟度	⭐⭐⭐⭐⭐	⭐⭐⭐⭐
生产验证	大量案例	AT&T、Vexxhost等
适合团队	传统运维团队	有K8s经验的团队
扩展性	好	优秀 (K8s生态加持)
与K8s共存	需独立K8s集群	天然共存

选型建议

选择 Kolla-Ansible 如果:
  ✅ 团队 Ansible 经验丰富，K8s 经验少
  ✅ 需要快速部署上线
  ✅ 纯虚拟化场景，不需要同时运行 K8s 工作负载
  ✅ 运维团队偏好简单直接的管理方式

选择 K8s + OpenStack-Helm 如果:
  ✅ 团队有 K8s 运维经验
  ✅ 需要统一管理 VM 和容器工作负载
  ✅ 看重自动扩缩容、自愈、滚动升级等 K8s 能力
  ✅ 有 GitOps / CI-CD 流水线需求
  ✅ 需要更精细的资源隔离和调度

13. 迁移策略

13.1 从 Kolla-Ansible 迁移到 K8s

┌─────────────────────────────────────────────────────────┐
│                   迁移路线图                              │
│                                                         │
│  Phase 1: 并行建设 (4-8 周)                               │
│  ├── 搭建 K8s 集群 (复用控制面硬件)                       │
│  ├── 部署 OpenStack-Helm (控制面服务)                     │
│  ├── 共享外部 Ceph 存储集群                               │
│  ├── 共享 OVN 网络 (或独立部署)                           │
│  └── 验证新平台功能                                       │
│                                                         │
│  Phase 2: 灰度切换 (4-8 周)                               │
│  ├── DNS 切换: 新 VM 创建走 K8s OpenStack               │
│  ├── 共享 Keystone (联邦认证) 或独立 Keystone              │
│  ├── 逐批迁移计算节点 (DaemonSet 注册到 K8s)              │
│  └── 验证业务连续性                                       │
│                                                         │
│  Phase 3: 旧平台下线 (2-4 周)                             │
│  ├── 停止 Kolla-Ansible 管理服务                          │
│  ├── 清理旧 Docker 容器                                  │
│  ├── 迁移剩余 VM 或就地升级                               │
│  └── 文档更新、运维培训                                   │
│                                                         │
└─────────────────────────────────────────────────────────┘

13.2 共享存储迁移

bash

# 两个 OpenStack 平台共享同一个 Ceph 集群
# Nova 实例和 Cinder 卷的 RBD image 天然共享

# 1. K8s OpenStack 创建实例时引用已有 RBD image
openstack image create "existing-vm-image" \
  --container-format bare \
  --disk-format raw \
  --property location=rbd:images/existing-image-id

# 2. Cinder 卷自动可见 (共享 ceph pool)
openstack volume list  # 可以看到 Kolla 平台创建的卷

# 3. Nova 实例迁移:
#    a. 在 Kolla 平台导出实例元数据
#    b. 在 K8s 平台重新创建实例 (指向相同 RBD image)

附录: 快速验证环境

最小验证环境 (3 节点)

bash

# 在 3 台 VM 上快速搭建验证环境
# 节点: ctrl-1, worker-1, worker-2

# 1. 用 kubeadm 初始化 K8s (单 master)
kubeadm init --pod-network-cidr=10.244.0.0/16

# 2. 安装 Calico
kubectl apply -f \
  https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml

# 3. 加入 worker 节点
kubeadm join ctrl-1:6443 --token xxx --discovery-token-ca-cert-hash sha256:xxx

# 4. 部署 OpenStack-Helm (最小化)
git clone https://opendev.org/openstack/openstack-helm.git
cd openstack-helm

# 5. 部署 Keystone (验证基本流程)
helm install keystone ./charts/keystone \
  --namespace openstack --create-namespace \
  --set pod.replicas.api=1 \
  --set pod.replicas.bootstrap=1

# 6. 验证
kubectl get pods -n openstack -w
kubectl logs -n openstack job/keystone-bootstrap

Kubernetes 部署 OpenStack 方案 ​

目录 ​

1. 方案概述 ​

1.1 核心理念 ​

1.2 适用场景 ​

2. 方案对比与选型 ​

2.1 三大方案对比 ​

2.2 推荐选型: OpenStack-Helm ​

3. 整体架构设计 ​

3.1 架构分层 ​

3.2 Namespace 规划 ​

3.3 节点角色设计 ​

4. Kubernetes 集群规划 ​

4.1 K8s 集群部署 (Kubespray) ​

4.2 K8s 集群核心配置 ​

4.3 部署 K8s 集群 ​

4.4 大规模 K8s 优化 ​

5. OpenStack-Helm 部署详解 ​

5.1 安装 Helm 和依赖 ​

5.2 部署顺序 ​

5.3 自动化部署脚本 ​

6. 基础设施服务部署 ​

6.1 MariaDB Galera Cluster ​

6.2 RabbitMQ Cluster ​

6.3 Memcached Cluster ​

7. OpenStack 核心服务部署 ​

7.1 Keystone ​

7.2 Nova (控制面 + 计算面分离) ​

7.3 Neutron (配合 OVN) ​

7.4 Glance (镜像服务) ​

7.5 Cinder (块存储) ​

8. 网络集成 (OVN) ​

8.1 OVN 在 K8s 上的部署 ​

8.2 多网络平面 (Multus CNI) ​

9. 存储集成 (Ceph) ​

9.1 方案一: Rook-Ceph (K8s 原生管理) ​

9.2 方案二: 外部 Ceph 集群集成 ​

9.3 Ceph RBD StorageClass ​

10. 高可用与故障恢复 ​

10.1 K8s 层面的 HA ​

10.2 Pod 反亲和性 (跨节点/跨机架) ​

10.3 PodDisruptionBudget ​

10.4 自动扩缩容 (HPA) ​

11. 运维管理 ​

11.1 监控 (Prometheus + Grafana) ​

11.2 日志收集 (Loki) ​

11.3 常用运维命令 ​

11.4 升级策略 ​

12. 与 Kolla-Ansible 对比 ​

选型建议 ​

13. 迁移策略 ​

13.1 从 Kolla-Ansible 迁移到 K8s ​

13.2 共享存储迁移 ​

附录: 快速验证环境 ​

最小验证环境 (3 节点) ​

Kubernetes 部署 OpenStack 方案

目录

1. 方案概述

1.1 核心理念

1.2 适用场景

2. 方案对比与选型

2.1 三大方案对比

2.2 推荐选型: OpenStack-Helm

3. 整体架构设计

3.1 架构分层

3.2 Namespace 规划

3.3 节点角色设计

4. Kubernetes 集群规划

4.1 K8s 集群部署 (Kubespray)

4.2 K8s 集群核心配置

4.3 部署 K8s 集群

4.4 大规模 K8s 优化

5. OpenStack-Helm 部署详解

5.1 安装 Helm 和依赖

5.2 部署顺序

5.3 自动化部署脚本

6. 基础设施服务部署

6.1 MariaDB Galera Cluster

6.2 RabbitMQ Cluster

6.3 Memcached Cluster

7. OpenStack 核心服务部署

7.1 Keystone

7.2 Nova (控制面 + 计算面分离)

7.3 Neutron (配合 OVN)

7.4 Glance (镜像服务)

7.5 Cinder (块存储)

8. 网络集成 (OVN)

8.1 OVN 在 K8s 上的部署

8.2 多网络平面 (Multus CNI)

9. 存储集成 (Ceph)

9.1 方案一: Rook-Ceph (K8s 原生管理)

9.2 方案二: 外部 Ceph 集群集成

9.3 Ceph RBD StorageClass

10. 高可用与故障恢复

10.1 K8s 层面的 HA

10.2 Pod 反亲和性 (跨节点/跨机架)

10.3 PodDisruptionBudget

10.4 自动扩缩容 (HPA)

11. 运维管理

11.1 监控 (Prometheus + Grafana)

11.2 日志收集 (Loki)

11.3 常用运维命令

11.4 升级策略

12. 与 Kolla-Ansible 对比

选型建议

13. 迁移策略

13.1 从 Kolla-Ansible 迁移到 K8s

13.2 共享存储迁移

附录: 快速验证环境

最小验证环境 (3 节点)