## 準備簽發證書環境 這個在運維主機HDSS-79.host.com上： #### 1、安裝CFSSL ``` wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O /usr/bin/cfssl wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O /usr/bin/cfssl-json wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -O /usr/bin/cfssl-certinfo chmod +x /usr/bin/cfssl* ``` 創建證書存放位置(可自己設置)： ``` mkdir /opt/certs cd /opt/certs ``` #### 2、創建生成CA證書簽名請求（csr）的JSON配置文件 ``` cat /opt/certs/ca-csr.json { "CN": "OldboyEdu", "hosts": [ ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "beijing", "L": "beijing", "O": "od", "OU": "ops" } ], "ca": { "expiry": "175200h" } } ``` #### 3、在/opt/certs生成CA證書和私鑰 ``` cfssl gencert -initca ca-csr.json | cfssl-json -bare ca ``` ## 安裝etcd 安裝機器HDSS7-12,HDSS7-21,HDSS7-22 #### 創建基于根證書的config配置文件 ``` cat ca-config.json { "signing": { "default": { "expiry": "175200h" }, "profiles": { "server": { "expiry": "175200h", "usages": [ "signing", "key encipherment", "server auth" ] }, "client": { "expiry": "175200h", "usages": [ "signing", "key encipherment", "client auth" ] }, "peer": { "expiry": "175200h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } ``` #### 創建生成自簽證書簽名請求（csr）的JION配置文件 ``` vi etcd-peer-csr.json { "CN": "k8s-etcd", "hosts": [ "10.4.7.11", "10.4.7.12", "10.4.7.21", "10.4.7.22" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "beijing", "L": "beijing", "O": "od", "OU": "ops" } ] } ``` #### 生成etcd證書和私鑰 ``` cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssl-json -bare etcd-peer ``` etcd下載地址：https://github.com/etcd-io/etcd/tags #### 解壓文件 ``` wget https://github.com/etcd-io/etcd/releases/download/v3.2.28/etcd-v3.2.28-linux-amd64.tar.gz tar -zxf etcd-v3.2.28-linux-amd64.tar.gz -C ../ ln -s /opt/etcd-v3.2.28-linux-amd64/ /opt/etcd mkdir -p /opt/etcd/certs /data/etcd /data/logs/etcd-server ``` #### 創建用戶 ``` useradd -s /sbin/nologin -M etcd id etcd ``` #### 證書發生到etcd機器 ``` scp ca.pem etcd-peer.pem etcd-peer-key.pem hdss7-12:/opt/etcd/certs ``` #### 啟動文件配置 ``` #!/bin/sh ./etcd --name etcd-server-7-12 \ --data-dir /data/etcd/etcd-server \ --listen-peer-urls https://10.4.7.12:2380 \ --listen-client-urls https://10.4.7.12:2379,http://127.0.0.1:2379 \ --quota-backend-bytes 8000000000 \ --initial-advertise-peer-urls https://10.4.7.12:2380 \ --advertise-client-urls https://10.4.7.12:2379,http://127.0.0.1:2379 \ --initial-cluster etcd-server-7-12=https://10.4.7.12:2380,etcd-server-7-21=https://10.4.7.21:2380,etcd-server-7-22=https://10.4.7.22:2380 \ --ca-file ./certs/ca.pem \ --cert-file ./certs/etcd-peer.pem \ --key-file ./certs/etcd-peer-key.pem \ --client-cert-auth \ --trusted-ca-file ./certs/ca.pem \ --peer-ca-file ./certs/ca.pem \ --peer-cert-file ./certs/etcd-peer.pem \ --peer-key-file ./certs/etcd-peer-key.pem \ --peer-client-cert-auth \ --peer-trusted-ca-file ./certs/ca.pem \ --log-output stdout ``` #### 執行文件給權限 ``` chmod +x /opt/etcd/etcd-server-startup.sh ``` #### 目錄授權（這個地方要住權限一定要給） ``` chown -R etcd.etcd /opt/etcd/certs /data/etcd /data/logs/etcd-server ``` #### Supervisor安裝部署 ``` yum install -y epel-release yum install -y supervisor systemctl enable supervisord # 開機自啟動 systemctl start supervisord # 啟動supervisord服務 ``` #### Supervisor配置etcd啟動文件 ``` vi /etc/supervisord.d/etcd-server.ini [program:etcd-server] command=/opt/etcd/etcd-server-startup.sh ; the program (relative uses PATH, can take args) numprocs=1 ; number of processes copies to start (def 1) directory=/opt/etcd ; directory to cwd to before exec (def no cwd) autostart=true ; start at supervisord start (default: true) autorestart=true ; retstart at unexpected quit (default: true) startsecs=30 ; number of secs prog must stay running (def. 1) startretries=3 ; max # of serial start failures (default 3) exitcodes=0,2 ; 'expected' exit codes for process (default 0,2) stopsignal=QUIT ; signal used to kill process (default TERM) stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10) user=etcd ; setuid to this UNIX account to run the program redirect_stderr=true ; redirect proc stderr to stdout (default false) stdout_logfile=/data/logs/etcd-server/etcd.stdout.log ; stdout log path, NONE for none; default AUTO stdout_logfile_maxbytes=64MB ; max # logfile bytes b4 rotation (default 50MB) stdout_logfile_backups=4 ; # of stdout logfile backups (default 10) stdout_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default 0) stdout_events_enabled=false ; emit events on stdout writes (default false) killasgroup=true stopasgroup=true ``` #### 更新supervisord ``` supervisorctl update supervisorctl status ``` #### 驗證etcd服務 ``` cd /opt/etcd/ ./etcdctl cluster-health ./etcdctl member list ```  ### 到這里etcd已經部署完成了 ### 4、里面其實存儲了很多的key，如何查看和管理這些key，需要使用一個小工具，叫做etcdkeeper： ### 下載etcd,在etcd節點選一臺 ``` cd /opt/src wget https://github.com/evildecay/etcdkeeper/releases/download/v0.7.5/etcdkeeper-v0.7.5-linux_x86_64.zip 解開壓縮包，需安裝unzip： yum install unzip -y unzip etcdkeeper-*-linux_x86_64.zip rm etcdkeeper-*-linux_x86_64.zip mv etcdkeeper ../ ln -s /opt/etcdkeeper-0.7.5/ /opt/etcdkeeper cd /opt/etcdkeeper chmod +x etcdkeeper ``` #### 編寫一個服務文件該服務文件主要用于在后臺運行etcd程序,用以提供http服務 ``` vi /lib/systemd/system/etcdkeeper.service [Unit] Description=etcdkeeper service After=network.target [Service] Type=simple ExecStart=/opt/etcdkeeper/etcdkeeper -h 10.4.7.12 -p 8800 ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=on-failure PrivateTmp=true [Install] WantedBy=multi-user.target ``` #### 服務的控制 ``` systemctl start etcdkeeper 啟動etcdkeeper服務 systemctl stop etcdkeeper 停止etcdkeeper服務 systemctl enable etcdkeeper.service 設置開機自啟動 systemctl disable etcdkeeper.service 停止開機自啟動 ``` #### 測試訪問http://10.4.7.12:8800/etcdkeeper