centos7部署etcd節點 · 運維心得

簡介：Etcd 是什么 * etcd是一個分布式可靠的鍵值存儲，用于分布式系統的最關鍵數據，重點是： 1.*簡單*：定義明確，面向用戶的API（gRPC） 2.*安全*：具有可選客戶端證書身份驗證的自動TLS 3.*快速*：基準測試10,000次/秒 4.*可靠*：使用Raft一致性算法分布集群 * etcd是用Go編寫的，使用[Raft](https://raft.github.io/)一致性算法來管理高度可用的復制日志。 * etcd被[許多公司](https://github.com/coreos/etcd/blob/master/Documentation/production-users.md)用于[生產](https://github.com/coreos/etcd/blob/master/Documentation/production-users.md)，開發團隊在關鍵部署場景中支持它，其中etcd經常與[Kubernetes](http://kubernetes.io/)，[locksmith](https://github.com/coreos/locksmith)，[vulcand](https://github.com/vulcand/vulcand)，[Doorman](https://github.com/youtube/doorman)等許多應用程序配合使用。通過**嚴格的測試**進一步確保可靠性。 * 有關簡單的命令行客戶端，請參閱[etcdctl](https://github.com/coreos/etcd/tree/master/etcdctl)(etcdctl --helper) ### 一、ETCD單機部署 ##### 1、下載二進制包 https://github.com/coreos/etcd/releases ``` wget https://github.com/etcd-io/etcd/releases/download/v3.3.12/etcd-v3.3.12-linux-arm64.tar.gz ``` ##### 2、解壓縮 ``` tar -zxvf etcd-v3.3.12-linux-arm64.tar.gz ``` ##### 3、設置環境變量 ``` export ETCDPATH="/home/etcd/etcd-v3.3.12-linux-amd64" export ETCDCTL_API=3 export PATH="$PATH:$ETCDPATH" ``` ##### 4、啟動etcd ``` etcd --data-dir $ETCDPATH/test_data ``` ##### 5、客戶端測試 ``` etcdctl --endpoints=http://127.0.0.1:2379 put foo bar etcdctl --endpoints=http://127.0.0.1:2379 get foo ``` ``` [root@bogon /]# etcdctl --endpoints=http://127.0.0.1:2379 put foo bar OK [root@bogon /]# etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar ``` ### 二、docker單機部署ETCD ##### 1、拉取etcd鏡像 ``` docker pull quay.io/coreos/etcd:v3.3.9 ``` ##### 2、設置環境變量和監聽地址 ``` -e ETCDCTL_API=3 -p 2379:2379 -p 2380:2380 ``` ##### 3、運行etcd容器 ``` docker run -d -it --rm \ --name etcd_test \ -e ETCDCTL_API=3 \ -p 2379:2379 \ -p 2380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd \ --advertise-client-urls http://0.0.0.0:2379 \ --listen-client-urls http://0.0.0.0:2379 ``` 容器退出自動刪除rm ``` [root@bogon etcd]# docker run -d -it --rm --name etcd_test -e ETCDCTL_API=3 -p 2379:2379 -p 2380:2380 quay.io/coreos/etcd:v3.3.9 etcd --advertise-client-urls http://0.0.0.0:2379 --listen-client-urls http://0.0.0.0:2379 ca444c094b6bf00b47726b0dee600620d3962cd0fb78d224db9fae12da94dc13 ``` 容器停止總是重啟 ``` [root@ansible-server ~]# docker run -d -it --restart=always --name etcd1 -e ETCDCTL_API=3 -p 2379:2379 -p 2380:2380 quay.io/coreos/etcd:v3.3.9 etcd --advertise-client-urls http://0.0.0.0:2379 --listen-client-urls http://0.0.0.0:2379 199b521dcdd1a742701350c8024dd76c179db2ded0d05a40dd44adfd14f2fb27 ``` 其他狀態： * no，默認策略，在容器退出時不重啟容器 * on-failure，在容器非正常退出時（退出狀態非0），才會重啟容器 * on-failure:3，在容器非正常退出時重啟容器，最多重啟3次 * always，在容器退出時總是重啟容器 * unless-stopped，在容器退出時總是重啟容器，但是不考慮在Docker守護進程啟動時就已經停止了的容器 ``` [root@bogon etcd]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ca444c094b6b quay.io/coreos/etcd:v3.3.9 "etcd --advertise-..." 4 seconds ago Up 3 seconds 0.0.0.0:2379-2380->2379-2380/tcp etcd_test ``` ##### 4、客戶端測試 ``` etcdctl --endpoints=http://127.0.0.1:2379 put foo bar etcdctl --endpoints=http://127.0.0.1:2379 get foo ``` ``` [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 put foo bar OK [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 put foo bar1 OK [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar1 ``` ### 三、Etcd本地集群（使用goreman管理） #### 1、下載etcd二進制包&&設置環境變量&&參考Etcd單機部署 #### 2、安裝goreman，安裝goreman前需要安裝go環境和git。 #### 3、安裝go環境 ``` wget https://studygolang.com/dl/golang/go1.10.1.linux-amd64.tar.gz tar -xvf go1.10.1.linux-amd64.tar.gz vim /etc/profile export GOROOT=/home/go/go export GOPATH=/home/go/data export PATH=$PATH:$GOROOT/bin:$GOPATH/bin source /etc/profile ``` #### 4、安裝git ``` yum -y install git ``` #### 5、安裝 goreman ``` go get github.com/mattn/goreman ``` 下載后的文件在```export GOPATH=/home/go/data```設置的路徑中 #### 6、編寫Procfile（管理集群） [https://github.com/coreos/etcd/blob/master/Procfile](https://github.com/coreos/etcd/blob/master/Procfile) #### 7、啟動etcd（goreman runlist） ``` goreman -f Procfile start ``` ### 四、docker cluster（集群） #### 8、etcd集群 ``` docker pull busybox ``` ##### docker netword ``` #docker network create etcd_cluster docker network create --subnet 172.16.3.0/16 etcd_cluster docker network inspect etcd_cluster #docker network rm etcd_cluster docker run -itd --rm --network etcd_cluster --ip 172.16.3.0 --name test busybox ``` ``` docker network ls docker network rm etcd_cluster ``` #### 9、etcd cluster ``` # etcd1 docker run -itd --restart=always \ --network etcd_cluster \ --ip 172.16.3.31 \ --hostname etcd1 \ --name etcd1 \ -e ETCDCTL_API=3 \ -p 12379:2379 \ -p 12380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd --name etcd1 \ --initial-advertise-peer-urls http://172.16.3.31:2380 \ --listen-peer-urls http://172.16.3.31:2380 \ --listen-client-urls http://172.16.3.31:2379,http://127.0.0.1:2379 \ --advertise-client-urls http://172.16.3.31:2379 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster etcd1=http://172.16.3.31:2380,etcd2=http://172.16.3.32:2380,etcd3=http://172.16.3.33:2380 \ --initial-cluster-state new # etcd2 docker run -itd --restart=always \ --network etcd_cluster \ --ip 172.16.3.32 \ --hostname etcd2 \ --name etcd2 \ -e ETCDCTL_API=3 \ -p 22379:2379 \ -p 22380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd --name etcd2 \ --initial-advertise-peer-urls http://172.16.3.32:2380 \ --listen-peer-urls http://172.16.3.32:2380 \ --listen-client-urls http://172.16.3.32:2379,http://127.0.0.1:2379 \ --advertise-client-urls http://172.16.3.32:2379 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster etcd1=http://172.16.3.31:2380,etcd2=http://172.16.3.32:2380,etcd3=http://172.16.3.33:2380 \ --initial-cluster-state new # etcd3 docker run -itd --restart=always \ --network etcd_cluster \ --ip 172.16.3.33 \ --hostname etcd3 \ --name etcd3 \ -e ETCDCTL_API=3 \ -p 32379:2379 \ -p 32380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd --name etcd3 \ --initial-advertise-peer-urls http://172.16.3.33:2380 \ --listen-peer-urls http://172.16.3.33:2380 \ --listen-client-urls http://172.16.3.33:2379,http://127.0.0.1:2379 \ --advertise-client-urls http://172.16.3.33:2379 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster etcd1=http://172.16.3.31:2380,etcd2=http://172.16.3.32:2380,etcd3=http://172.16.3.33:2380 \ --initial-cluster-state new # client #進入容器 docker exec -it etcd1 bin/sh #客戶端測試 etcdctl --write-out=table --endpoints=http://127.0.0.1:2379 member list etcdctl --endpoints=http://127.0.0.1:2379 put foo bar etcdctl --endpoints=http://127.0.0.1:2379 get foo #測試結果如下 [root@bogon ~]# docker exec -it etcd1 bin/sh / # etcdctl --write-out=table --endpoints=http://127.0.0.1:2379 member list +------------------+---------+-------+-------------------------+-------------------------+ | ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | +------------------+---------+-------+-------------------------+-------------------------+ | c26d6ba798c079c | started | etcd3 | http://172.16.3.33:2380 | http://172.16.3.33:2379 | | 4631df2115e1ef72 | started | etcd2 | http://172.16.3.32:2380 | http://172.16.3.32:2379 | | a92fe5422902bc40 | started | etcd1 | http://172.16.3.31:2380 | http://172.16.3.31:2379 | +------------------+---------+-------+-------------------------+-------------------------+ / # etcdctl --endpoints=http://127.0.0.1:2379 put foo bar OK / # etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar #docker ps [root@bogon docker]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5988c4443eb5 quay.io/coreos/etcd:v3.3.9 "etcd --name etcd3..." 37 seconds ago Up 36 seconds 0.0.0.0:32379->2379/tcp, 0.0.0.0:32380->2380/tcp etcd3 d4fdda6400cc quay.io/coreos/etcd:v3.3.9 "etcd --name etcd2..." 44 seconds ago Up 44 seconds 0.0.0.0:22379->2379/tcp, 0.0.0.0:22380->2380/tcp etcd2 af86e5fd2ae5 quay.io/coreos/etcd:v3.3.9 "etcd --name etcd1..." 53 seconds ago Up 52 seconds 0.0.0.0:2379-2380->2379-2380/tcp etcd1 ```