一、自簽APIServer SSL證書
1、 進入TLS目錄
1.1、執行shell:
```
generate_k8s_cert.sh
```
shell命令的內容是:
```
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
```
1.2、 server-csr.json
node節點不用配置
```
{
??? "CN": "kubernetes",
??? "hosts": [
????? "10.0.0.1",
????? "127.0.0.1",
????? "kubernetes",
????? "kubernetes.default",
????? "kubernetes.default.svc",
????? "kubernetes.default.svc.cluster",
????? "kubernetes.default.svc.cluster.local",
????? "192.168.72.166",
????? "192.168.72.167",
????? "192.168.72.170",
????? "192.168.72.171",
????? "192.168.72.172"
??? ],
??? "key": {
??????? "algo": "rsa",
??????? "size": 2048
??? },
??? "names": [
??????? {
??????????? "C": "CN",
??????????? "L": "BeiJing",
??????????? "ST": "BeiJing",
??????????? "O": "k8s",
??????????? "OU": "System"
??????? }
??? ]
}
```
1.3、 kube-proxy-csr.json
```
{
? "CN": "system:kube-proxy",
? "hosts": [],
? "key": {
??? "algo": "rsa",
??? "size": 2048
? },
? "names": [
??? {
????? "C": "CN",
????? "L": "BeiJing",
????? "ST": "BeiJing",
????? "O": "k8s",
????? "OU": "System"
??? }
? ]
}
```
2、生產證書
```
./generate_k8s_cert.sh
```
二、部署Master組件
1、下載
https://github.com/kubernetes/kubernetes/releases/tag/v1.17.4
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.17.md#server-binaries
https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/
2、Master組件
2.1、 kube-apiserver
1) kube-apiserver.service
```
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-apiserver.conf
ExecStart=/opt/kubernetes/bin/kube-apiserver $KUBE_APISERVER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
2)kube-apiserver.conf
```
KUBE_APISERVER_OPTS="--logtostderr=false \
--v=2 \
--log-dir=/opt/kubernetes/logs \
--etcd-servers=https://192.168.254.201:2379,https://192.168.254.203:2379,https://192.168.254.204:2379 \
--bind-address=192.168.254.201 \ #監聽的地址
--secure-port=6443 \
--advertise-address=192.168.254.201 \ #通告的地址
--allow-privileged=true \ #所有權限
--service-cluster-ip-range=10.0.0.0/24 \ #service的ip范圍
--service-node-port-range=1-65535 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \ #準入的插件
--authorization-mode=RBAC,Node \ #授權的模式
--enable-bootstrap-token-auth=true \ #自動頒發證書
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-32767 \ #暴露的端口范圍
--kubelet-client-certificate=/opt/kubernetes/ssl/server.pem \
--kubelet-client-key=/opt/kubernetes/ssl/server-key.pem \
--tls-cert-file=/opt/kubernetes/ssl/server.pem? \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem \
--audit-log-maxage=30 \
--audit-log-maxbackup=3 \
--audit-log-maxsize=100 \
--audit-log-path=/opt/kubernetes/logs/k8s-audit.log"
```
2.2、kube-controller-manager
1)kube-controller-manager
```
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-controller-manager.conf
ExecStart=/opt/kubernetes/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
2)kube-controller-manager.conf
```
KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=false \
--v=2 \
--log-dir=/opt/kubernetes/logs \
--leader-elect=true \ #集群的選舉
--master=127.0.0.1:8080 \
--address=127.0.0.1 \
--allocate-node-cidrs=true \
--cluster-cidr=10.244.0.0/16 \
--service-cluster-ip-range=10.0.0.0/24?\
--cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \
--cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem? \
--root-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \
--experimental-cluster-signing-duration=87600h0m0s"
```
2.3、 kube-scheduler
1)kube-scheduler.service
```
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-scheduler.conf
ExecStart=/opt/kubernetes/bin/kube-scheduler $KUBE_SCHEDULER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target
```
2)kube-scheduler.conf
```
KUBE_SCHEDULER_OPTS="--logtostderr=false \
--v=2 \
--log-dir=/opt/kubernetes/logs \
--leader-elect \
--master=127.0.0.1:8080 \
--address=127.0.0.1"
```
3、實驗
3.1、上傳tar并且解壓
3.2、拷貝證書到ssl里面
我們拷貝生成的證書到kubernetes中的ssl文件中
```
cp /yhj/TLS/k8s/*.pem /yhj/kubernetes/ssl/
```

3.3、cp -rf kubernetes /opt

3.4、拷貝service到執行目錄下
```
cp kube-apiserver.service kube-controller-manager.service kube-scheduler.service /usr/lib/systemd/system
```

3.5、啟動
```
# systemctl start kube-apiserver
# systemctl start kube-controller-manager
# systemctl start kube-scheduler
# systemctl enable kube-apiserver
# systemctl enable kube-controller-manager
# systemctl enable kube-scheduler
#systemctl restart kube-apiserver
for i in $(ls /opt/kubernetes/bin);do systemctl enable $i;done
```
3.6、日志查看
```
less /opt/kubernetes/logs
```
```
cd /opt/kubernetes/logs
```

```
less kube-apiserver.INFO
```
3.7、開機啟動
subtopic
3.8、kubectl get node
3.9、查看
```
cd /opt/kubernetes/bin/
```

```
./kubectl get cs
```

3.10、驗證
```
ps -ef | grep kube
ps -ef | grep kube | wc -4
```
4、 授權啟用TLS Bootstrapping
4.1、文件拷貝

4.2、cat /opt/kubernetes/cfg/token.csv
格式:token,用戶,uid,用戶組
c47ffb939f5ca36231d9e3121a252940,kubelet-bootstrap,10001,"system:node-bootstrapper"
4.3、給kubelet-bootstrap授權:
```
kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap
```
4.4、token也可自行生成替換:
```
head -c 16 /dev/urandom | od -An -t x | tr -d ' '
```
注意:我們如果需要最新的,我們只需要下載最新版,然后替換掉bin目錄中的文件即可