# 華為云cce 部署 多節點nacos ## 在k8s中部署的難點 在k8s中部署nacos集群和在裸機器上直接部署nacos機器其實差別不大。 最主要的區別是k8s中部署的服務沒有固定的ip地址，而nacos集群部署需要配置所有實例的ip ## 2 解決 * 在k8s中通過StatefulSet和Headless Service為每個nacos實例生成一個唯一的dns地址， * 創建一個普通Service給可客戶端使用 ## 3 實現方式 ### 3.1 創建數據庫配置 ———————————————— ``` --- apiVersion: v1 kind: Secret metadata: name: nacos type: Opaque data: mysql.db.host: "NDFjZTgyZjY4OWE5NGU4ZDk4YmRlOWQ2MDQ2ZTA0YWRpbjAxLmludGVybmFsLmNuLWVhc3QtMy5teXNxbC5yZHMubXlodWF3ZWljbG91ZC5jb20=" mysql.db.name: "bmFjb3M=" mysql.db.port: "MzMwNg==" mysql.db.user: "bmFjb3M=" mysql.db.password: "V0ExNmdvVWE2bU5oUmlqRg==" --- ``` > Opaque Secret Opaque 類型的數據是一個 map 類型，要求value是base64編碼格式。 比如我們來創建一個用戶名為 nacos, 的 Secret 對象， 首先我們先把這用戶名和密碼做 base64 編碼 ``` $ echo -n 'nacos' | openssl base64 bmFjb3M= ``` ### 3.2 部署Headless Service ———————————————— Headless Service為每個pod（nacos實例）生成一個DNS地址，用作NACOS\_SERVERS配置 ``` --- apiVersion: v1 kind: Service metadata: name: nacos-headless labels: app: nacos annotations: service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" spec: ports: - port: 8848 name: server targetPort: 8848 - port: 9848 name: client-rpc targetPort: 9848 - port: 9849 name: raft-rpc targetPort: 9849 ## 兼容1.4.x版本的選舉端口 - port: 7848 name: old-raft-rpc targetPort: 7848 clusterIP: None selector: app: nacos --- ``` ### 3.3 通過StatefulSet部署nacos StatefulSet部署方式為每個POD生成固定的名稱，如nacos-0、nacos-1、nacos-2等。 ``` apiVersion: apps/v1 kind: StatefulSet metadata: name: nacos spec: serviceName: nacos-headless replicas: 3 template: metadata: labels: app: nacos annotations: pod.alpha.kubernetes.io/initialized: "true" spec: volumes: - name: vol-163912341665228473 hostPath: path: /opt/logs/ type: '' affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: "app" operator: In values: - nacos topologyKey: "kubernetes.io/hostname" initContainers: - name: peer-finder-plugin-install image: nacos/nacos-peer-finder-plugin:1.1 imagePullPolicy: IfNotPresent volumeMounts: - mountPath: /home/nacos/plugins/peer-finder name: data subPath: peer-finder containers: - name: nacos imagePullPolicy: Always image: swr.cn-east-3.myhuaweicloud.com/huyi-base/nacos-server:2.0.3 resources: limits: cpu: '2' memory: 4Gi requests: memory: '4Gi' cpu: '2' ports: - containerPort: 8848 name: client-port - containerPort: 9848 name: client-rpc - containerPort: 9849 name: raft-rpc - containerPort: 7848 name: old-raft-rpc env: - name: NACOS_REPLICAS value: "3" - name: SERVICE_NAME value: "nacos-headless" - name: DOMAIN_NAME value: "cluster.local" - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: MYSQL_SERVICE_HOST valueFrom: secretKeyRef: name: nacos key: mysql.db.host - name: MYSQL_SERVICE_DB_NAME valueFrom: secretKeyRef: name: nacos key: mysql.db.name - name: MYSQL_SERVICE_PORT valueFrom: secretKeyRef: name: nacos key: mysql.db.port - name: MYSQL_SERVICE_USER valueFrom: secretKeyRef: name: nacos key: mysql.db.user - name: MYSQL_SERVICE_PASSWORD valueFrom: secretKeyRef: name: nacos key: mysql.db.password - name: NACOS_SERVER_PORT value: "8848" - name: NACOS_APPLICATION_PORT value: "8848" - name: PREFER_HOST_MODE value: "hostname" volumeMounts: - name: data mountPath: /home/nacos/plugins/peer-finder subPath: peer-finder - name: data mountPath: /home/nacos/data subPath: data - name: vol-163912341665228473 mountPath: /home/nacos/logs/ policy: logs: rotate: Hourly annotations: format: '{"multi":{"mode":"time","value":"YYYY-MM-DD hh:mm:ss"}}' pathPattern: nacos.log imagePullSecrets: - name: default-secret volumeClaimTemplates: - metadata: name: data annotations: everest.io/disk-volume-type: SAS labels: failure-domain.beta.kubernetes.io/region: cn-east-3 failure-domain.beta.kubernetes.io/zone: cn-east-3a spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 20Gi storageClassName: csi-disk selector: matchLabels: app: nacos ``` ### 初始化建表 mysql [MySQL 建表](https://github.com/alibaba/nacos/blob/master/distribution/conf/nacos-mysql.sql)