K8S
1、项目总览
1.1、最终架构方案
1.2、IP分配规划
1.3、域名规划
1.4、CPU优化指南
1.5、云C部署指南
1.6、部署检查清单
1.7、快速参考手册
2.1、API-VIP高可用配置
2.2、Calico网络配置
2.3、存储方案配置
2.4、Ingress入口配置
2.5、安全加固配置
2.6、etcd优化配置
2.7、灾难恢复配置
2.8、公司网络配置
K8s部署
本文档使用 MrDoc 发布
-
+
首页
1.3、域名规划
# 域名规划方案 ## 🌐 域名分层设计 ### 设计原则 - **管理与业务分离**:避免运维域名与业务域名混淆 - **安全隔离**:管理域名仅内网访问,业务域名公网暴露 - **灵活扩展**:支持多业务线、多环境 --- ## 📋 域名分配方案 ### 1. ktnet.cc - K8s运维管理专用域名 **用途**:集群管理、监控、CI/CD、镜像仓库等**内部服务** **访问限制**: - ✅ 仅通过WireGuard内网访问 - ✅ 公司办公网通过VPN访问 - ❌ 禁止公网直接访问 #### 子域名分配 | 域名 | 服务 | 解析目标 | 端口 | 说明 | |------|------|---------|------|------| | **api.ktnet.cc** | K8s API Server | 10.255.0.100 (VIP) | 6443 | 集群管理入口 | | **dashboard.ktnet.cc** | K8s Dashboard | Ingress ClusterIP | 443 | Web管理界面 | | **harbor.ktnet.cc** | Harbor镜像仓库 | 172.16.72.10 | 443 | 容器镜像管理 | | **monitor.ktnet.cc** | Grafana监控 | 172.16.72.11 | 443 | 监控Dashboard | | **alert.ktnet.cc** | AlertManager | 172.16.72.11 | 443 | 告警管理 | | **logs.ktnet.cc** | Kibana/Grafana Loki | 172.16.72.13 | 443 | 日志查询 | | **git.ktnet.cc** | Gitea | 10.255.1.12 | 443 | Git代码仓库 | | **running.ktnet.cc** | Running | 10.255.1.16 | 443 | CI/CD流水线 | | **argocd.ktnet.cc** | ArgoCD | Ingress ClusterIP | 443 | GitOps部署 | | **minio.ktnet.cc** | MinIO Console | 172.16.72.20 | 9001 | 对象存储管理 | | **prometheus.ktnet.cc** | Prometheus | 172.16.72.11 | 9090 | 指标查询 | **DNS配置示例:** ```bash # CoreDNS Hosts配置(内网解析) apiVersion: v1 kind: ConfigMap metadata: name: coredns namespace: kube-system data: Corefile: | ktnet.cc:53 { hosts { 10.255.0.100 api.ktnet.cc 10.96.50.100 dashboard.ktnet.cc 172.16.72.10 harbor.ktnet.cc 172.16.72.11 monitor.ktnet.cc 172.16.72.11 alert.ktnet.cc 172.16.72.13 logs.ktnet.cc 172.16.72.12 ci.ktnet.cc 10.96.50.101 argocd.ktnet.cc 172.16.72.20 minio.ktnet.cc 172.16.72.11 prometheus.ktnet.cc ttl 60 } log } ``` **Ingress配置示例:** ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ktnet-management namespace: kube-system annotations: cert-manager.io/cluster-issuer: "ktnet-selfsigned" nginx.ingress.kubernetes.io/ssl-redirect: "true" # 白名单:仅允许内网访问 nginx.ingress.kubernetes.io/whitelist-source-range: "10.255.0.0/24,172.16.72.0/21,192.168.72.0/24" spec: ingressClassName: nginx-internal # 使用内网Ingress tls: - hosts: - dashboard.ktnet.cc - argocd.ktnet.cc secretName: ktnet-wildcard-tls rules: - host: dashboard.ktnet.cc http: paths: - path: / pathType: Prefix backend: service: name: kubernetes-dashboard port: number: 443 - host: argocd.ktnet.cc http: paths: - path: / pathType: Prefix backend: service: name: argocd-server port: number: 443 ``` --- ### 2. ktcloud.cc - 企业业务服务专用域名 **用途**:所有对外提供的业务服务 **访问限制**: - ✅ 公网可访问 - ✅ 通过CDN加速(可选) - ✅ 支持Let's Encrypt自动证书 #### 二级域名命名规范 **格式:** `[env.]<service>.ktcloud.cc` - `env`:环境前缀(prod/gray/test,生产环境可省略) - `service`:业务服务名称 #### 业务域名示例 | 业务线 | 生产域名 | 灰度域名 | 测试域名 | 说明 | |-------|---------|---------|---------|------| | **电商平台** | ec.ktcloud.cc | gray.ec.ktcloud.cc | test.ec.ktcloud.cc | 电商主站 | | **支付服务** | pay.ktcloud.cc | gray.pay.ktcloud.cc | test.pay.ktcloud.cc | 支付网关 | | **用户中心** | user.ktcloud.cc | gray.user.ktcloud.cc | test.user.ktcloud.cc | 用户管理 | | **后台管理** | admin.ktcloud.cc | gray.admin.ktcloud.cc | test.admin.ktcloud.cc | 后台系统 | | **API网关** | api.ktcloud.cc | gray.api.ktcloud.cc | test.api.ktcloud.cc | API统一入口 | | **移动端API** | m.ktcloud.cc | gray.m.ktcloud.cc | test.m.ktcloud.cc | 移动端接口 | | **静态资源** | static.ktcloud.cc | - | - | CDN静态资源 | | **WebSocket** | ws.ktcloud.cc | gray.ws.ktcloud.cc | test.ws.ktcloud.cc | 实时通信 | **公网DNS配置(以Cloudflare为例):** ```bash # A记录(指向MetalLB分配的公网IP) ec.ktcloud.cc A 172.93.107.95 # 生产(云A IP池) gray.ec.ktcloud.cc A 172.93.107.109 # 灰度(云A IP池) test.ec.ktcloud.cc A 172.93.107.138 # 测试(云B IP池) pay.ktcloud.cc A 172.93.107.96 # 生产 gray.pay.ktcloud.cc A 172.93.107.109 # 灰度 test.pay.ktcloud.cc A 172.93.107.138 # 测试 # 泛域名(未来扩展,可选) *.ktcloud.cc A 172.93.107.95 # 默认指向生产 *.gray.ktcloud.cc A 172.93.107.109 # 灰度泛域名 *.test.ktcloud.cc A 172.93.107.138 # 测试泛域名 ``` **Ingress配置示例:** ```yaml # 生产环境业务Ingress apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: production-services namespace: production annotations: cert-manager.io/cluster-issuer: "letsencrypt-prod" nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/rate-limit: "100" # 限流 # 绑定MetalLB生产IP池 metallb.universe.tf/address-pool: "prod-pool" spec: ingressClassName: nginx-public # 使用公网Ingress tls: - hosts: - ec.ktcloud.cc - pay.ktcloud.cc - user.ktcloud.cc - admin.ktcloud.cc - api.ktcloud.cc secretName: ktcloud-prod-tls # Let's Encrypt自动证书 rules: - host: ec.ktcloud.cc http: paths: - path: / pathType: Prefix backend: service: name: ecommerce-service port: number: 80 - host: pay.ktcloud.cc http: paths: - path: / pathType: Prefix backend: service: name: payment-service port: number: 80 - host: api.ktcloud.cc http: paths: - path: / pathType: Prefix backend: service: name: api-gateway port: number: 80 --- # 灰度环境业务Ingress apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: staging-services namespace: staging annotations: cert-manager.io/cluster-issuer: "letsencrypt-prod" nginx.ingress.kubernetes.io/ssl-redirect: "true" # 灰度流量控制(Canary部署) nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-weight: "20" # 20%流量到灰度 # 绑定MetalLB灰度IP池 metallb.universe.tf/address-pool: "gray-pool" spec: ingressClassName: nginx-public tls: - hosts: - gray.ec.ktcloud.cc - gray.pay.ktcloud.cc secretName: ktcloud-gray-tls rules: - host: gray.ec.ktcloud.cc http: paths: - path: / pathType: Prefix backend: service: name: ecommerce-service-gray port: number: 80 ``` --- ## 🔐 证书管理策略 ### 1. ktnet.cc - 自签名证书(内网) ```yaml # 创建自签名CA apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: ktnet-selfsigned spec: selfSigned: {} --- # 申请泛域名证书 apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: ktnet-wildcard namespace: kube-system spec: secretName: ktnet-wildcard-tls issuerRef: name: ktnet-selfsigned kind: ClusterIssuer dnsNames: - "ktnet.cc" - "*.ktnet.cc" duration: 87600h # 10年 renewBefore: 720h # 提前30天续期 ``` **证书分发:** - 导出CA根证书:`kubectl get secret ktnet-wildcard-tls -n kube-system -o jsonpath='{.data.ca\.crt}' | base64 -d > ktnet-ca.crt` - 安装到办公电脑:Windows信任根证书、Mac钥匙串、Linux `/etc/ssl/certs/` --- ### 2. ktcloud.cc - Let's Encrypt证书(公网) ```yaml # 配置Cloudflare DNS-01验证(支持泛域名) apiVersion: v1 kind: Secret metadata: name: cloudflare-api-token namespace: cert-manager type: Opaque stringData: api-token: <Cloudflare API Token> --- # Let's Encrypt生产环境Issuer apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: email: ops@ktnet.cc server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: letsencrypt-prod-account-key solvers: # DNS-01验证(支持泛域名) - dns01: cloudflare: apiTokenSecretRef: name: cloudflare-api-token key: api-token selector: dnsNames: - "*.ktcloud.cc" - "*.gray.ktcloud.cc" - "*.test.ktcloud.cc" # HTTP-01验证(单域名备用) - http01: ingress: class: nginx-public selector: dnsNames: - "ec.ktcloud.cc" - "pay.ktcloud.cc" --- # 申请泛域名证书 apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: ktcloud-wildcard namespace: default spec: secretName: ktcloud-wildcard-tls issuerRef: name: letsencrypt-prod kind: ClusterIssuer dnsNames: - "ktcloud.cc" - "*.ktcloud.cc" - "*.gray.ktcloud.cc" - "*.test.ktcloud.cc" duration: 2160h # 90天 renewBefore: 720h # 提前30天自动续期 ``` --- ## 🌍 MetalLB公网IP池分配 ### IP池与域名环境对应关系 ```yaml # 生产IP池(云A) apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: prod-pool namespace: metallb-system spec: addresses: - 172.93.107.95-172.93.107.107 # 13个IP autoAssign: false # 手动分配 --- # 灰度IP池(云A+云B) apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: gray-pool namespace: metallb-system spec: addresses: - 172.93.107.109/32 # 云A - 172.93.107.136/32 # 云B autoAssign: false --- # 测试IP池(云B+云C) apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: test-pool namespace: metallb-system spec: addresses: - 172.93.107.138/32 # 云B - <云C IP1>/32 # 云C(待分配) autoAssign: false --- # L2广播配置 apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: business-l2 namespace: metallb-system spec: ipAddressPools: - prod-pool - gray-pool - test-pool nodeSelectors: - matchLabels: node-role.kubernetes.io/worker: "" ``` --- ## 📝 域名申请与配置清单 ### 部署前准备 #### 1. 域名注册(如已完成可跳过) - [x] ktnet.cc(运维管理) - [x] ktcloud.cc(业务服务) #### 2. DNS托管(推荐Cloudflare) - [ ] 将ktnet.cc NS记录指向Cloudflare - [ ] 将ktcloud.cc NS记录指向Cloudflare - [ ] 获取Cloudflare API Token(用于cert-manager DNS-01验证) #### 3. 内网DNS配置 - [ ] 配置CoreDNS Hosts(ktnet.cc内网解析) - [ ] 测试内网域名解析:`nslookup api.ktnet.cc 10.96.0.10` #### 4. 公网DNS配置 - [ ] 添加ktcloud.cc A记录(指向MetalLB IP) - [ ] 配置泛域名(可选) - [ ] 测试公网域名解析:`nslookup ec.ktcloud.cc 8.8.8.8` #### 5. 证书配置 - [ ] 部署cert-manager - [ ] 创建ktnet.cc自签名证书 - [ ] 创建ktcloud.cc Let's Encrypt证书 - [ ] 导出并安装ktnet.cc根证书到办公电脑 --- ## 🚀 部署顺序 ```bash # 第1步:部署cert-manager kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml # 第2步:创建Cloudflare Secret kubectl create secret generic cloudflare-api-token \ --from-literal=api-token=<YOUR_TOKEN> \ -n cert-manager # 第3步:创建ClusterIssuer kubectl apply -f - <<EOF apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: ktnet-selfsigned spec: selfSigned: {} --- apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: email: ops@ktnet.cc server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: letsencrypt-prod-key solvers: - dns01: cloudflare: apiTokenSecretRef: name: cloudflare-api-token key: api-token EOF # 第4步:申请证书 kubectl apply -f ktnet-cert.yaml kubectl apply -f ktcloud-cert.yaml # 第5步:验证证书 kubectl get certificate -A kubectl describe certificate ktnet-wildcard -n kube-system kubectl describe certificate ktcloud-wildcard -n default ``` --- ## 🔍 验证清单 ### 内网域名验证(ktnet.cc) ```bash # 1. 从公司办公网访问 curl -k https://api.ktnet.cc:6443/healthz # 预期:返回 "ok" curl -k https://harbor.ktnet.cc # 预期:返回Harbor登录页面 curl -k https://monitor.ktnet.cc # 预期:返回Grafana登录页面 # 2. 证书验证 openssl s_client -connect harbor.ktnet.cc:443 -showcerts # 预期:看到自签名证书链 ``` ### 公网域名验证(ktcloud.cc) ```bash # 1. DNS解析验证 nslookup ec.ktcloud.cc # 预期:解析到 172.93.107.95 dig ec.ktcloud.cc +short # 预期:返回公网IP # 2. HTTPS访问验证 curl -I https://ec.ktcloud.cc # 预期:返回200 OK,证书为Let's Encrypt # 3. 证书有效性验证 echo | openssl s_client -connect ec.ktcloud.cc:443 2>/dev/null | openssl x509 -noout -dates # 预期:看到有效期90天,自动续期 ``` --- ## 📊 域名规划总结 | 域名 | 用途 | 访问方式 | 证书类型 | IP池 | |------|------|---------|---------|------| | **ktnet.cc** | K8s运维管理 | 仅内网/VPN | 自签名(10年) | WireGuard内网 | | **ktcloud.cc** | 企业业务服务 | 公网 | Let's Encrypt(90天自动续期) | MetalLB公网IP | **安全隔离:** - ✅ 运维人员通过VPN访问 ktnet.cc 管理集群 - ✅ 最终用户直接访问 ktcloud.cc 使用业务服务 - ✅ 两套域名完全隔离,互不影响 - ✅ ktnet.cc不暴露到公网,提升安全性 --- **下一步:** 1. 完成DNS配置 2. 申请Cloudflare API Token 3. 部署cert-manager 4. 申请并验证证书
arise
2025年11月22日 09:49
转发文档
收藏文档
‹‹
‹
4
/ 17
›
››
手机扫码
复制链接
手机扫一扫转发分享
复制链接
Markdown文件
PDF文档(打印)
分享
链接
类型
密码
更新密码