## 運維痛處 k8s集群在線上跑了一段時間，可是我發現我對集群內部的變化沒有辦法把控的很清楚，好比某個pod被從新調度了、某個node節點上的imagegc失敗了、某個hpa被觸發了等等，而這些都是能夠經過events拿到的，可是events并非永久存儲的，它包含了集群各類資源的狀態變化，因此咱們能夠經過收集分析events來了解整個集群內部的變化。 ## kubernetes-event-exporter產品 [該存儲庫](https://github.com/opsgenie/kubernetes-event-exporter) 該工具允許將經常錯過的 Kubernetes 事件導出到各種輸出，以便將它們用于可觀察性或警報目的。你不會相信你錯過了資源的變化。 ## 安裝 kubernetes-event-exporter > 提請創建好 `kube-mon` 命名空間 1. 相關的rbac權限 ```yaml --- apiVersion: v1 kind: ServiceAccount metadata: namespace: kube-mon name: event-exporter --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: event-exporter roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: view subjects: - kind: ServiceAccount namespace: kube-mon name: event-exporter ``` 2. events配置文件 ```yaml apiVersion: v1 kind: ConfigMap metadata: name: event-exporter-cfg namespace: kube-mon data: config.yaml: | logLevel: error logFormat: json route: routes: - match: - receiver: "dump" receivers: - name: "dump" ## kafka配置 kafka: clientId: "kubernetes" topic: "kube-events" brokers: - "192.168.31.235:9092" ## elasticsearch配置 ## 必須要有index、indexFormat參數 #elasticsearch: # hosts: # - http://192.168.31.235:9200 # index: kube-events # indexFormat: "kube-events-{2006-01-02}" # useEventID: true ## 直接輸出到容器日志 # stdout: {} ``` 更多的配置內容請查看 [官方文檔](https://github.com/opsgenie/kubernetes-event-exporter#configuration) 3. events資源清單 ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: event-exporter namespace: kube-mon spec: replicas: 1 template: metadata: labels: app: event-exporter version: v1 spec: serviceAccountName: event-exporter containers: - name: event-exporter image: ghcr.io/opsgenie/kubernetes-event-exporter:v0.11 imagePullPolicy: IfNotPresent args: - -conf=/data/config.yaml volumeMounts: - mountPath: /data name: cfg volumes: - name: cfg configMap: name: event-exporter-cfg selector: matchLabels: app: event-exporter version: v1 ``` 4. 創建以上的清單 ```shell $ kubectl apply -f 00-roles.yaml serviceaccount/event-exporter created clusterrolebinding.rbac.authorization.k8s.io/event-exporter created $ kubectl apply -f 01-config.yaml configmap/event-exporter-cfg created $ kubectl apply -f 02-deployment.yaml deployment.apps/event-exporter created ``` ## 驗證 ```shell $ kubectl -n kube-mon get deploy event-exporter NAME READY UP-TO-DATE AVAILABLE AGE event-exporter 1/1 1 1 85s $ kafka-get-offsets.sh --bootstrap-server 192.168.31.235:9092 --topic kube-events kube-events:0:43 ``` > 手動刪除一個pod。查看topic為 `kube-events` 的返回結果，返回結果為 `kube-events:0:43`，分別是topic名稱、partitionID、partitionID的偏移量。查看任意partitionID的偏移量是否為0。 > - 0 --> 代表沒有上傳到Kafka > - 非0 --> 代表有數據上傳到Kafka，說明整個流程正常