[TOC]
# 必要條件
- Kubernetes 必須配置為使用 CNI(請參閱 [網絡插件要求](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#network-plugin-requirements))
- Linux kernel >= 4.9.17,升級內核 [參考步驟](../../other/upgrade_kernel.md)
- 卸載其他cni插件,請參閱 [卸載calico網絡插件](../calico/uninstall.md)
- 有關系統要求的更多詳細信息,請參閱 [系統要求](https://docs.cilium.io/en/stable/operations/system_requirements/#admin-system-reqs)
# 與kubernets兼容性
| cilium version | k8s Version |
| :-: | :-: |
| v1.11 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23 |
| v1.12 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23, 1.24 |
| v1.13 | 1.16, 1.17, 1.18, 1.19, 1.20, 1.21, 1.22, 1.23, 1.24, 1.25, 1.26 |
| v1.14 | 1.19, 1.20, 1.21, 1.22, 1.23, 1.24, 1.25, 1.26, 1.27 |
| v1.15 | 1.26, 1.27, 1.28, 1.29 |
>[info] 參考:https://docs.cilium.io/en/v1.13/network/kubernetes/requirements/
# 安裝cilium
## cilium cli方式
cilium CLI命令安裝
```shell
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; elif [ "$(uname -m)" = "x86_64" ]; then CLI_ARCH=amd64; fi
curl -L -O https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm -f cilium-linux-${CLI_ARCH}.tar.gz
```
安裝cilium
```shell
$ cilium install --version 1.13.12
?? Using Cilium version 1.13.12
Auto-detected cluster name: kubernetes
Auto-detected kube-proxy has been installed
```
>[info] 查看 cilium 所有版本 `cilium install --list-versions`
驗證cilium
```shell
$ cilium status --wait
/ˉˉ\
/ˉˉ\__/ˉˉ\ Cilium: OK
\__/ˉˉ\__/ Operator: OK
/ˉˉ\__/ˉˉ\ Envoy DaemonSet: disabled (using embedded mode)
\__/ˉˉ\__/ Hubble Relay: disabled
\__/ ClusterMesh: disabled
Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1
DaemonSet cilium Desired: 3, Ready: 3/3, Available: 3/3
Containers: cilium Running: 3
cilium-operator Running: 1
Cluster Pods: 0/7 managed by Cilium
Helm chart version: 1.13.12
Image versions cilium-operator quay.io/cilium/operator-generic:v1.13.12@sha256:f83734bbe270f961d545c7929152785507ce04a05d818ebc9776941723736d02: 1
cilium quay.io/cilium/cilium:v1.13.12@sha256:d99204aa7b3b7bd2c9ab47fd398cc9f40290799bc0c7a4386c8dc5c1780cd3d3: 3
```
## helm方式【推薦】
```shell
$ helm repo add cilium https://helm.cilium.io/
"cilium" has been added to your repositories
$ helm install cilium cilium/cilium --version 1.13.12 \
--namespace kube-system \
--set operator.replicas=1 \
--set k8sServiceHost=192.168.32.100 \
--set k8sServicePort=6443
NAME: cilium
LAST DEPLOYED: Fri Mar 22 10:35:58 2024
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.
Your release version is 1.13.12.
For any further help, visit https://docs.cilium.io/en/v1.13/gettinghelp
```
驗證cilium
```shell
$ kubectl -n kube-system get pod | grep cilium
cilium-h5tb5 1/1 Running 0 23s
cilium-n6sjg 1/1 Running 0 23s
cilium-operator-6b465dcf6d-q7dtm 1/1 Running 0 23s
cilium-vfl4d 1/1 Running 0 23s
```
# cilium開啟hubble功能
Hubble 是 Cilium 的可觀察層,可用于獲取 Kubernetes 集群的網絡和安全層的集群范圍內的可見性
## cilium cli方式
開啟Hubble功能
```shell
$ cilium hubble enable --ui
```
>[info] 啟用 Hubble 需要在運行 Cilium 的所有節點上打開 TCP 端口 4244。 這是 hubble-relay 正確運行所必需的。
運行 cilium status 以驗證 Hubble 是否已啟用并正在運行
```shell
$ cilium status
/ˉˉ\
/ˉˉ\__/ˉˉ\ Cilium: OK
\__/ˉˉ\__/ Operator: OK
/ˉˉ\__/ˉˉ\ Envoy DaemonSet: disabled (using embedded mode)
\__/ˉˉ\__/ Hubble Relay: OK
\__/ ClusterMesh: disabled
Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1
Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1
Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1
DaemonSet cilium Desired: 3, Ready: 3/3, Available: 3/3
Containers: cilium Running: 3
hubble-relay Running: 1
cilium-operator Running: 1
hubble-ui Running: 1
Cluster Pods: 7/7 managed by Cilium
Helm chart version: 1.13.12
Image versions cilium quay.io/cilium/cilium:v1.13.12@sha256:d99204aa7b3b7bd2c9ab47fd398cc9f40290799bc0c7a4386c8dc5c1780cd3d3: 3
hubble-relay quay.io/cilium/hubble-relay:v1.13.12@sha256:01b23ea40bcd81145dde6bfcbfc4d542749d08c2a1c6348954c85123a8d2b1fc: 1
cilium-operator quay.io/cilium/operator-generic:v1.13.12@sha256:f83734bbe270f961d545c7929152785507ce04a05d818ebc9776941723736d02: 1
hubble-ui quay.io/cilium/hubble-ui:v0.13.0@sha256:7d663dc16538dd6e29061abd1047013a645e6e69c115e008bee9ea9fef9a6666: 1
hubble-ui quay.io/cilium/hubble-ui-backend:v0.13.0@sha256:1e7657d997c5a48253bb8dc91ecee75b63018d16ff5e5797e5af367336bc8803: 1
```
## helm方式
修改安裝cilium參數
```shell
# 獲取當前版本號
cilium_version=$(helm -n kube-system ls | awk '/cilium/ {print $NF}')
echo $cilium_version
# 備份上一次安裝的參數
helm -n kube-system get values cilium > cilium_custom.yaml
sed -i '1d' cilium_custom.yaml
# 添加安裝Hubble參數
cat <<EOF | tee -a cilium_custom.yaml >> /dev/null
hubble:
relay:
enabled: true
ui:
enabled: true
EOF
# 修改Hubble參數
helm -n kube-system upgrade cilium cilium/cilium --version $cilium_version -f cilium_custom.yaml
```
回滾版本
```shell
# 查看所有版本
$ helm -n kube-system history cilium
# 回滾到上一個版本
$ helm -n kube-system rollback cilium
Rollback was a success! Happy Helming!
# 回滾到指定版本
$ helm -n kube-system rollback cilium 9
Rollback was a success! Happy Helming!
```
# 安裝hubble cli
```shell
HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; elif [ "$(uname -m)" = "x86_64" ]; then HUBBLE_ARCH=amd64; fi
curl -L -O https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz
sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin
rm -f hubble-linux-${HUBBLE_ARCH}.tar.gz
```
## 驗證 Hubble API 訪問
1. 暴露hubble-relay服務
>[info] 默認hubble-relay服務是沒有暴露端口的。hubble命令需要訪問hubble-relay服務,所以需要把服務暴露出來
```shell
$ cilium hubble port-forward &
[1] 31717
$ kubectl port-forward svc/hubble-relay -n kube-system 4245:80 &
[1] 31425
```
2. 驗證是否可以通過安裝的 CLI 訪問 Hubble API
```shell
$ hubble status
Healthcheck (via localhost:4245): Ok
Current/Max Flows: 12,285/12,285 (100.00%)
Flows/s: 89.72
Connected Nodes: 3/3
```
3. 命令行查看網絡調用流
```shell
$ hubble observe
```
## 驗證 Hubble UI 訪問
```shell
# 只暴露127.0.0.1地址
$ cilium hubble ui
# 監聽所有網卡地址
$ kubectl port-forward -n kube-system svc/hubble-ui --address 0.0.0.0 12000:80
```

- 前言
- 架構
- 部署
- kubeadm部署
- kubeadm擴容節點
- 二進制安裝基礎組件
- 添加master節點
- 添加工作節點
- 選裝插件安裝
- Kubernetes使用
- k8s與dockerfile啟動參數
- hostPort與hostNetwork異同
- 應用上下線最佳實踐
- 進入容器命名空間
- 主機與pod之間拷貝
- events排序問題
- k8s會話保持
- 容器root特權
- CNI插件
- calico
- calicoctl安裝
- calico網絡通信
- calico更改pod地址范圍
- 新增節點網卡名不一致
- 修改calico模式
- calico數據存儲遷移
- 啟用 kubectl 來管理 Calico
- calico卸載
- cilium
- cilium架構
- cilium/hubble安裝
- cilium網絡路由
- IP地址管理(IPAM)
- Cilium替換KubeProxy
- NodePort運行DSR模式
- IP地址偽裝
- ingress使用
- nginx-ingress
- ingress安裝
- ingress高可用
- helm方式安裝
- 基本使用
- Rewrite配置
- tls安全路由
- ingress發布管理
- 代理k8s集群外的web應用
- ingress自定義日志
- ingress記錄真實IP地址
- 自定義參數
- traefik-ingress
- traefik名詞概念
- traefik安裝
- traefik初次使用
- traefik路由(IngressRoute)
- traefik中間件(middlewares)
- traefik記錄真實IP地址
- cert-manager
- 安裝教程
- 頒布者CA
- 創建證書
- 外部存儲
- 對接NFS
- 對接ceph-rbd
- 對接cephfs
- 監控平臺
- Prometheus
- Prometheus安裝
- grafana安裝
- Prometheus配置文件
- node_exporter安裝
- kube-state-metrics安裝
- Prometheus黑盒監控
- Prometheus告警
- grafana儀表盤設置
- 常用監控配置文件
- thanos
- Prometheus
- Sidecar組件
- Store Gateway組件
- Querier組件
- Compactor組件
- Prometheus監控項
- grafana
- Querier對接grafana
- alertmanager
- Prometheus對接alertmanager
- 日志中心
- filebeat安裝
- kafka安裝
- logstash安裝
- elasticsearch安裝
- elasticsearch索引生命周期管理
- kibana安裝
- event事件收集
- 資源預留
- 節點資源預留
- imagefs與nodefs驗證
- 資源預留 vs 驅逐 vs OOM
- scheduler調度原理
- Helm
- Helm安裝
- Helm基本使用
- 安全
- apiserver審計日志
- RBAC鑒權
- namespace資源限制
- 加密Secret數據
- 服務網格
- 備份恢復
- Velero安裝
- 備份與恢復
- 常用維護操作
- container runtime
- 拉取私有倉庫鏡像配置
- 拉取公網鏡像加速配置
- runtime網絡代理
- overlay2目錄占用過大
- 更改Docker的數據目錄
- Harbor
- 重置Harbor密碼
- 問題處理
- 關閉或開啟Harbor的認證
- 固定harbor的IP地址范圍
- ETCD
- ETCD擴縮容
- ETCD常用命令
- ETCD數據空間壓縮清理
- ingress
- ingress-nginx header配置
- kubernetes
- 驗證yaml合法性
- 切換KubeProxy模式
- 容器解析域名
- 刪除節點
- 修改鏡像倉庫
- 修改node名稱
- 升級k8s集群
- 切換容器運行時
- apiserver接口
- 其他
- 升級內核
- k8s組件性能分析
- ETCD
- calico
- calico健康檢查失敗
- Harbor
- harbor同步失敗
- Kubernetes
- 資源Terminating狀態
- 啟動容器報錯