kubeadm部署 · Kubernetes

[TOC] ## 修改主機名 ```shell sudo hostnamectl set-hostname <HOSTNAME> ``` >[info] `<HOSTNAME>` 根據實際的填寫 ## 停止防火墻 ```shell sudo systemctl stop firewalld && sudo systemctl disable firewalld sudo setenforce 0 sudo sed -ri 's/^(SELINUX)=.*$/\1=disabled/' /etc/selinux/config ``` ## 關閉swap分區 ```shell sudo swapoff -a sudo sed -i '/ swap / s/^$.*$$/#\1/g' /etc/fstab ``` ## 設置內核模塊 ```shell cat <<-EOF | sudo tee /etc/sysconfig/modules/ipvs.modules > /dev/null #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack modprobe -- br_netfilter EOF sudo chmod 755 /etc/sysconfig/modules/ipvs.modules && sudo bash /etc/sysconfig/modules/ipvs.modules ``` ## 設置的 sysctl 參數 ```shell cat <<-EOF | sudo tee /etc/sysctl.d/kubernetes.conf > /dev/null # 二層的網橋在轉發包時也會被iptables的FORWARD規則所過濾 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 # kernel轉發功能 net.ipv4.ip_forward = 1 # 允許將TIME-WAIT sockets重新用于新的TCP連接，默認為0，表示關閉 net.ipv4.tcp_tw_reuse = 1 # TCP連接中TIME-WAIT sockets的快速回收, 默認是0，表示關閉。對于位于NAT設備(容器轉發)后面的Client來說，就是一場災難 net.ipv4.tcp_tw_recycle = 0 # 允許系統打開的端口范圍，即用于向外連接的端口范圍 net.ipv4.ip_local_port_range = 32768 65535 # kernel中最多存在的TIME_WAIT數量, 默認是4096 net.ipv4.tcp_max_tw_buckets = 65535 # 控制系統是否開啟對數據包源地址的校驗(0 不校驗) net.ipv4.conf.all.rp_filter = 0 # 開啟ipv6路由轉發 net.ipv6.conf.all.forwarding = 1 # 開啟ipv4路由轉發 net.ipv4.conf.all.forwarding = 1 # FIN-WAIT-2狀態保持時間 net.ipv4.tcp_fin_timeout = 15 EOF sudo sysctl -p /etc/sysctl.d/kubernetes.conf ``` ## 安裝containerd 請參考博客園 [containerd安裝](https://www.cnblogs.com/jiaxzeng/p/16708491.html) 文章 ## 添加 k8s 源 ```shell cat <<-EOF | sudo tee /etc/yum.repos.d/kubernetes.repo > /dev/null [kubernetes] name=Aliyun-kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=0 EOF sudo yum clean all && sudo yum makecache ``` ## 安裝軟件 ``` sudo yum install -y kubelet-1.23.3-0.x86_64 kubeadm-1.23.3-0.x86_64 kubectl-1.23.3-0.x86_64 ipvsadm conntrack-tools sudo systemctl enable kubelet.service ``` >[info] 查看 kubeadm 所有版本 `yum list kubeadm --showduplicates` ## 拉取k8s組件鏡像 ```shell kubeadm config images pull --kubernetes-version=v1.23.3 --image-repository registry.aliyuncs.com/google_containers ``` ## 初始化集群配置 ```yaml --- apiVersion: kubeadm.k8s.io/v1beta3 kind: InitConfiguration nodeRegistration: criSocket: /run/containerd/containerd.sock --- apiVersion: kubeadm.k8s.io/v1beta3 kind: ClusterConfiguration kubernetesVersion: 1.23.3 controlPlaneEndpoint: 172.139.20.100 imageRepository: registry.aliyuncs.com/google_containers networking: serviceSubnet: 10.96.0.0/12 podSubnet: 172.26.0.0/16 --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration mode: ipvs ``` ## 初始化集群 ```shell sudo kubeadm init --config kubeadm-config.yaml --upload-certs --node-name 主機IP地址 ``` >[warning] **注意**：containerd 包含 cni 相關命令和配置可能影響安裝。 > *確認方法*：查看 `/opt/cni/bin` 和 `/etc/cni/net.d` 是否有文件。確認時非空目錄的話，需要刪除后在執行創建集群 ## 安裝calico插件 ### 下載manifest文件 ```shell mkdir /etc/kubernetes/addons curl https://projectcalico.docs.tigera.io/archive/v3.23/manifests/calico-etcd.yaml -o /etc/kubernetes/addons/calico.yaml ``` ### 生成calico證書 >[info] etcd證書在master節點上才有。 ```shell pki_dir=/etc/kubernetes/pki/etcd mkdir -p ${pki_dir} && cd ${pki_dir} openssl genrsa -out calico-etcd-client.key 2048 cat <<-EOF | sudo tee calico-etcd-client-csr.conf > /dev/null [ req ] default_bits = 2048 prompt = no default_md = sha256 distinguished_name = dn [ dn ] C = CN ST = Guangdong L = Guangzhou CN = calico-etcd-client [ v3_ext ] keyUsage=Digital Signature, Key Encipherment extendedKeyUsage=clientAuth basicConstraints=CA:FALSE authorityKeyIdentifier=keyid:always EOF openssl req -new -key calico-etcd-client.key -out calico-etcd-client.csr -config calico-etcd-client-csr.conf openssl x509 -req -in calico-etcd-client.csr -CA /etc/kubernetes/pki/etcd/ca.crt -CAkey /etc/kubernetes/pki/etcd/ca.key \ -CAcreateserial -out calico-etcd-client.crt -days 36500 \ -extensions v3_ext -extfile calico-etcd-client-csr.conf -sha256 openssl verify -CAfile /etc/kubernetes/pki/etcd/ca.crt ${pki_dir}/calico-etcd-client.crt [ $? -eq 0 ] && rm -rf ${pki_dir}/{calico-etcd-client-csr.conf,calico-etcd-client.csr} ``` ### 修改calico配置文件 1. 修改calico連接etcd的地址 >[info] 修改etcd地址 ```bash sed -ri 's@http://<ETCD_IP>:<ETCD_PORT>@https://192.168.32.127:2379,https://192.168.32.128:2379,https://192.168.32.129:2379@g' /etc/kubernetes/addons/calico.yaml ``` 2. 修改calico連接etcd證書 ```bash ETCD_CA=$(cat /etc/kubernetes/pki/etcd/ca.crt | base64 -w 0) ETCD_CERT=$(cat /etc/kubernetes/pki/etcd/calico-etcd-client.crt | base64 -w 0) ETCD_KEY=$(cat /etc/kubernetes/pki/etcd/calico-etcd-client.key | base64 -w 0) sed -ri "s@# etcd-ca: null@etcd-ca: ${ETCD_CA}@g" /etc/kubernetes/addons/calico.yaml sed -ri "s@# etcd-cert: null@etcd-cert: ${ETCD_CERT}@g" /etc/kubernetes/addons/calico.yaml sed -ri "s@# etcd-key: null@etcd-key: ${ETCD_KEY}@g" /etc/kubernetes/addons/calico.yaml sed -ri 's@etcd_ca: ""@etcd_ca: "/calico-secrets/etcd-ca"@g' /etc/kubernetes/addons/calico.yaml sed -ri 's@etcd_cert: ""@etcd_cert: "/calico-secrets/etcd-cert"@g' /etc/kubernetes/addons/calico.yaml sed -ri 's@etcd_key: ""@etcd_key: "/calico-secrets/etcd-key"@g' /etc/kubernetes/addons/calico.yaml ``` 3. 設置calico網段地址將 DaemonSet 類型，calico-node 的環境變量 `CALICO_IPV4POOL_CIDR` >[info] 默認是192.168.0.0/16地址 ```yaml - name: CALICO_IPV4POOL_CIDR value: "10.244.0.0/16" ``` ## 卸載集群 ```shell kubectl drain <node name> --delete-emptydir-data --force --ignore-daemonsets kubeadm reset -f rm -rf /etc/cni/net.d iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X ipvsadm -C kubectl delete node <node name> ```