[TOC] # 必要條件 - Kubernetes 必須配置為使用 CNI（請參閱 [網絡插件要求](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#network-plugin-requirements)） - Linux kernel >= 4.9.17，升級內核 [參考步驟](../../other/upgrade_kernel.md) - 卸載其他cni插件，請參閱 [卸載calico網絡插件](../calico/uninstall.md) - 有關系統要求的更多詳細信息，請參閱 [系統要求](https://docs.cilium.io/en/stable/operations/system_requirements/#admin-system-reqs) # 與kubernets兼容性 | cilium version | k8s Version | | :-: | :-: | | v1.11 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23 | | v1.12 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23, 1.24 | | v1.13 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23, 1.24, 1.25, 1.26 | | v1.14 | 1.19, 1.20, 1.21, 1.22, 1.23, 1.24, 1.25, 1.26, 1.27 | | v1.15 | 1.26, 1.27, 1.28, 1.29 | >[info] 參考：https://docs.cilium.io/en/v1.13/network/kubernetes/requirements/ # 安裝cilium ## cilium cli方式 cilium CLI命令安裝 ```shell CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt) if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; elif [ "$(uname -m)" = "x86_64" ]; then CLI_ARCH=amd64; fi curl -L -O https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin rm -f cilium-linux-${CLI_ARCH}.tar.gz ``` 安裝cilium ```shell $ cilium install --version 1.13.12 ?? Using Cilium version 1.13.12 Auto-detected cluster name: kubernetes Auto-detected kube-proxy has been installed ``` >[info] 查看 cilium 所有版本 `cilium install --list-versions` 驗證cilium ```shell $ cilium status --wait /ˉˉ\ /ˉˉ\__/ˉˉ\ Cilium: OK \__/ˉˉ\__/ Operator: OK /ˉˉ\__/ˉˉ\ Envoy DaemonSet: disabled (using embedded mode) \__/ˉˉ\__/ Hubble Relay: disabled \__/ ClusterMesh: disabled Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 DaemonSet cilium Desired: 3, Ready: 3/3, Available: 3/3 Containers: cilium Running: 3 cilium-operator Running: 1 Cluster Pods: 0/7 managed by Cilium Helm chart version: 1.13.12 Image versions cilium-operator quay.io/cilium/operator-generic:v1.13.12@sha256:f83734bbe270f961d545c7929152785507ce04a05d818ebc9776941723736d02: 1 cilium quay.io/cilium/cilium:v1.13.12@sha256:d99204aa7b3b7bd2c9ab47fd398cc9f40290799bc0c7a4386c8dc5c1780cd3d3: 3 ``` ## helm方式【推薦】 ```shell $ helm repo add cilium https://helm.cilium.io/ "cilium" has been added to your repositories $ helm install cilium cilium/cilium --version 1.13.12 \ --namespace kube-system \ --set operator.replicas=1 \ --set k8sServiceHost=192.168.32.100 \ --set k8sServicePort=6443 NAME: cilium LAST DEPLOYED: Fri Mar 22 10:35:58 2024 NAMESPACE: kube-system STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: You have successfully installed Cilium with Hubble. Your release version is 1.13.12. For any further help, visit https://docs.cilium.io/en/v1.13/gettinghelp ``` 驗證cilium ```shell $ kubectl -n kube-system get pod | grep cilium cilium-h5tb5 1/1 Running 0 23s cilium-n6sjg 1/1 Running 0 23s cilium-operator-6b465dcf6d-q7dtm 1/1 Running 0 23s cilium-vfl4d 1/1 Running 0 23s ``` # cilium開啟hubble功能 Hubble 是 Cilium 的可觀察層，可用于獲取 Kubernetes 集群的網絡和安全層的集群范圍內的可見性 ## cilium cli方式 開啟Hubble功能 ```shell $ cilium hubble enable --ui ``` >[info] 啟用 Hubble 需要在運行 Cilium 的所有節點上打開 TCP 端口 4244。 這是 hubble-relay 正確運行所必需的。 運行 cilium status 以驗證 Hubble 是否已啟用并正在運行 ```shell $ cilium status /ˉˉ\ /ˉˉ\__/ˉˉ\ Cilium: OK \__/ˉˉ\__/ Operator: OK /ˉˉ\__/ˉˉ\ Envoy DaemonSet: disabled (using embedded mode) \__/ˉˉ\__/ Hubble Relay: OK \__/ ClusterMesh: disabled Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1 Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1 Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1 DaemonSet cilium Desired: 3, Ready: 3/3, Available: 3/3 Containers: cilium Running: 3 hubble-relay Running: 1 cilium-operator Running: 1 hubble-ui Running: 1 Cluster Pods: 7/7 managed by Cilium Helm chart version: 1.13.12 Image versions cilium quay.io/cilium/cilium:v1.13.12@sha256:d99204aa7b3b7bd2c9ab47fd398cc9f40290799bc0c7a4386c8dc5c1780cd3d3: 3 hubble-relay quay.io/cilium/hubble-relay:v1.13.12@sha256:01b23ea40bcd81145dde6bfcbfc4d542749d08c2a1c6348954c85123a8d2b1fc: 1 cilium-operator quay.io/cilium/operator-generic:v1.13.12@sha256:f83734bbe270f961d545c7929152785507ce04a05d818ebc9776941723736d02: 1 hubble-ui quay.io/cilium/hubble-ui:v0.13.0@sha256:7d663dc16538dd6e29061abd1047013a645e6e69c115e008bee9ea9fef9a6666: 1 hubble-ui quay.io/cilium/hubble-ui-backend:v0.13.0@sha256:1e7657d997c5a48253bb8dc91ecee75b63018d16ff5e5797e5af367336bc8803: 1 ``` ## helm方式 修改安裝cilium參數 ```shell # 獲取當前版本號 cilium_version=$(helm -n kube-system ls | awk '/cilium/ {print $NF}') echo $cilium_version # 備份上一次安裝的參數 helm -n kube-system get values cilium > cilium_custom.yaml sed -i '1d' cilium_custom.yaml # 添加安裝Hubble參數 cat <<EOF | tee -a cilium_custom.yaml >> /dev/null hubble: relay: enabled: true ui: enabled: true EOF # 修改Hubble參數 helm -n kube-system upgrade cilium cilium/cilium --version $cilium_version -f cilium_custom.yaml ``` 回滾版本 ```shell # 查看所有版本 $ helm -n kube-system history cilium # 回滾到上一個版本 $ helm -n kube-system rollback cilium Rollback was a success! Happy Helming! # 回滾到指定版本 $ helm -n kube-system rollback cilium 9 Rollback was a success! Happy Helming! ``` # 安裝hubble cli ```shell HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt) if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; elif [ "$(uname -m)" = "x86_64" ]; then HUBBLE_ARCH=amd64; fi curl -L -O https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin rm -f hubble-linux-${HUBBLE_ARCH}.tar.gz ``` ## 驗證 Hubble API 訪問 1. 暴露hubble-relay服務 >[info] 默認hubble-relay服務是沒有暴露端口的。hubble命令需要訪問hubble-relay服務，所以需要把服務暴露出來 ```shell $ cilium hubble port-forward & [1] 31717 $ kubectl port-forward svc/hubble-relay -n kube-system 4245:80 & [1] 31425 ``` 2. 驗證是否可以通過安裝的 CLI 訪問 Hubble API ```shell $ hubble status Healthcheck (via localhost:4245): Ok Current/Max Flows: 12,285/12,285 (100.00%) Flows/s: 89.72 Connected Nodes: 3/3 ``` 3. 命令行查看網絡調用流 ```shell $ hubble observe ``` ## 驗證 Hubble UI 訪問 ```shell # 只暴露127.0.0.1地址 $ cilium hubble ui # 監聽所有網卡地址 $ kubectl port-forward -n kube-system svc/hubble-ui --address 0.0.0.0 12000:80 ```