Skip to main content

KubeSphere

version: v4.3.0, debian 11

Install​

required:

apt install -y ebtables ethtool socat conntrack iptables chrony
sudo systemctl enable --now chrony

kubekey​

export KKZONE=cn
curl -sfL https://get-kk.kubesphere.io | SKIP_PACKAGE=true sh -

inventory.yaml​

只安装本机

# vim inventory.yaml
apiVersion: kubekey.kubesphere.io/v1
kind: Inventory
metadata:
  name: default
spec:
  hosts:
    localhost:
      connector:
        type: local
        password: 1234 # 本地 sudo 密码
  groups:
    # 所有 Kubernetes 节点
    k8s_cluster:
      groups:
        - kube_control_plane
        - kube_worker
    # 控制平面节点
    kube_control_plane:
      hosts:
        - localhost
    # 工作节点
    kube_worker:
      hosts:
        - localhost
    # etcd 节点
    etcd:
      hosts:
        - localhost

k8s config​

如果要使用 cn 镜像，那么则必须从 构建离线包 的生成配置文件的地址去找可用的 k8s 版本：https://get-images.kubesphere.io

因为 hub.kubesphere.com.cn 并没有镜像全量的版本依赖。

./kk create config --with-kubernetes v1.33.7 -o .
# 修改为镜像从 cn 地区下载
spec:
  zone: "cn"

create cluster​

./kk create cluster -i inventory.yaml -c config-v1.33.7.yaml

install kubesphere-system​

chart=oci://hub.kubesphere.com.cn/kse/ks-core
version=1.2.5
helm upgrade --install -n kubesphere-system --create-namespace ks-core $chart \
  --debug --wait --version $version --reset-values --take-ownership \
  --set global.imageRegistry=hub.kubesphere.com.cn,extension.imageRegistry=hub.kubesphere.com.cn

gateway & nip.io​

requires: Gateway API:1.0(Traefik)

info

仍然在找最佳实践

  1. 在集群中 -> Gateway API -> 网关实现 -> 启用
  2. 在应用管理中部署 OpenELB（分配 IP）
  3. 替换加速镜像: admission.image.repository: docker.1ms.run/kubesphere/kube-webhook-certgen, 其他类似。
  4. 在 service 中，spec.type: LoadBalancer.
  5. 添加 annotations:
metadata:
  annotations:
    # kubesphere
    kubesphere.io/annotations: OpenELB
    # 从 eip-pool 里拿 ip
    eip.openelb.kubesphere.io/v1alpha2: eip-pool
    lb.kubesphere.io/v1alpha1: openelb
    protocol.openelb.kubesphere.io/v1alpha1: vip
# OpenELB 分配 IPs
apiVersion: network.kubesphere.io/v1alpha2
kind: Eip
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"network.kubesphere.io/v1alpha2","kind":"Eip","metadata":{"annotations":{},"name":"eip-pool"},"spec":{"address":"192.168.10.200-192.168.10.210","disable":false,"interface":"eth0","protocol":"vip"}}
  creationTimestamp: '2026-08-21T07:07:40Z'
  finalizers:
    - finalizer.ipam.kubesphere.io/v1alpha1
  name: eip-pool
spec:
  address: 192.168.10.200-192.168.10.210
  interface: eth0
  protocol: vip
# 不是一个好的方案
1. 在集群中 -> Gateway API -> 网关实现 -> 启用
2. 在企业空间中 -> 服务与网络 -> Gateway API -> 创建
3. 选择第 1 步中创建的网关 -> 域名 -> app-name.{host.ip}.nip.io 


kubectl get svc -n kubesphere-controls-system | grep gateway

#!/usr/bin/env bash
# Forward node:80 -> Traefik Gateway ClusterIP:80 (via DNAT + masquerade)
set -e

NODE_IP=192.168.10.189
TRAEFIK_IP=10.233.50.29

# Insert DNAT rule into nat PREROUTING (before cali/KUBE jumps), if absent
nft -a list chain ip nat PREROUTING 2>/dev/null | grep -q "daddr $NODE_IP.*dport 80" || \
  nft insert rule ip nat PREROUTING ip daddr $NODE_IP tcp dport 80 dnat to $TRAEFIK_IP:80

# Ensure masquerade for DNAT'd traffic in POSTROUTING
nft list chain ip nat POSTROUTING 2>/dev/null | grep -q 'ct status dnat' || \
  nft insert rule ip nat POSTROUTING ct status dnat masquerade

Trouble Shooting​

Secret "kubeconfig-admin" not found​

Failed to get KubeSphere version: the server is currently unable to handle the request (get services http:ks-apiserver:80)
# 查看系统状态
kubectl get pods -n kubesphere-system
kubectl logs -n kubesphere-system -l app=ks-apiserver --tail=100
# 检查默认 namespace 或 kubesphere-system 下是否存在根 kubeconfig Secret
kubectl get secret kubeconfig-admin -n kubesphere-system

# 如果不存在，使用当前节点的 /etc/kubernetes/admin.conf 创建它
kubectl create secret generic kubeconfig-admin \
  --from-file=kubeconfig=/etc/kubernetes/admin.conf \
  -n kubesphere-system
# 查看 host 集群的定义
kubectl get cluster host -o yaml
# 在补全 Secret 之后，重启相关服务触发重新加载：
kubectl rollout restart deployment/ks-apiserver -n kubesphere-system
kubectl rollout restart deployment/ks-controller-manager -n kubesphere-system

Pods Operation Unauthorized​

这个问题发生在 K8s 集群被暂停 24h 左右导致 Calico CNI token 过期。

  • calico-node pod 启动（init 写了初版 kubeconfig）
  • calico-node 某次容器重启 → 重新签发令牌（就是当前文件里这枚，iat 此刻）
  • calico-node 容器再次重启（第 3 次）
  • 令牌到期 → 此后任何 pod 网络操作都会 Unauthorized
  • calico-node 每天 ~06:10 左右会重启一次，它每次重启都重新签发一枚 24h 令牌——只要它天天重启，令牌就天天续上。

可一旦某次重启没赶上、或过期后没立刻再重启，令牌就变 stale，CNI 立刻挂掉。

calico-node 写进 kubeconfig 的是一枚只有 24h 有效期的令牌，且没有任何滚动续期机制。它靠“calico-node 每天重启顺手重签”勉强续命，这是个脆弱设计。

debian 11​

apt mirror​

# /etc/apt/source.list
# deb cdrom:[Debian GNU/Linux 11.11.0 _Bullseye_ - Official amd64 DVD Binary-1 20240831-14:01]/ bullseye contrib main

# 默认注释了源码镜像以提高 apt update 速度，如有需要可自行取消注释
deb https://mirror.nju.edu.cn/debian/ bullseye main contrib non-free
# deb-src https://mirror.nju.edu.cn/debian/ bullseye main contrib non-free

deb https://mirror.nju.edu.cn/debian/ bullseye-updates main contrib non-free
# deb-src https://mirror.nju.edu.cn/debian/ bullseye-updates main contrib non-free

# deb https://mirror.nju.edu.cn/debian/ bullseye-backports main contrib non-free
# deb-src https://mirror.nju.edu.cn/debian/ bullseye-backports main contrib non-free

# 以下安全更新软件源为官方源配置
deb https://mirror.nju.edu.cn/debian-security bullseye-security main contrib non-free
# deb-src https://security.debian.org/debian-security bullseye-security main contrib non-free
apt install -y curl wget sudo vim git zsh sudo isc-dhcp-client

static network​

# /etc/network/interfaces

auto eth0
iface eth0 inet static
    address 192.168.10.189/24
    gateway 192.168.10.1
    dns-nameservers 192.168.10.1

DNS

# /etc/resolv.conf

nameserver 192.168.10.1
systemctl restart networking

Extra​

# 1. 停止并清理集群
./kk delete cluster -i inventory.yaml

# 2. 清理残余的 K8s 配置与 etcd 目录
sudo kubeadm reset -f
sudo rm -rf /etc/kubernetes /var/lib/etcd /var/lib/kubelet $HOME/.kube /etc/cni/net.d

# 3. 重启容器运行时（如果用的是 containerd）
sudo systemctl restart containerd

refs:​