namespace資源限制 · Kubernetes

[TOC] 資源配額，通過 ResourceQuota 對象來定義，對每個命名空間的資源消耗總量提供限制。它可以限制命名空間中某種類型的對象的總數目上限，也可以限制命令空間中的 Pod 可以使用的計算資源的總上限。 ## 啟用資源配額資源配額的支持在很多 Kubernetes 版本中是默認啟用的。當 API 服務器的命令行標志 --enable-admission-plugins= 中包含 ResourceQuota 時，資源配額會被啟用。 ```shell grep enable-admission-plugins /data/k8s/conf/kube-apiserver.conf ``` ## 限制資源配置 ```yaml apiVersion: v1 kind: ResourceQuota metadata: name: quota namespace: test spec: hard: requests.cpu: "4" requests.memory: "1Gi" requests.storage: "100Gi" pods: "4" ``` > **說明：** > 如果所使用的是 CRI 容器運行時，容器日志會被計入臨時存儲配額。這可能會導致存儲配額耗盡的 Pods 被意外地驅逐出節點。參考日志架構了解詳細信息。 ## 生效資源限制 ```shell $ kubectl apply -f test-namespaces.yml resourcequota/quota create ``` ## 驗證 ### 驗證內存創建一個 `1Gi` 的deployment清單文件 ```yaml apiVersion: apps/v1 kind: Deployment metadata: labels: app: client name: client spec: replicas: 1 selector: matchLabels: app: client strategy: {} template: metadata: labels: app: client spec: containers: - image: busybox:1.24.1 name: busybox args: - sh - -c - "sleep 3600" resources: requests: memory: 2Gi ``` 創建容器 ```shell $ kubectl apply -f client.yml -n test deployment.apps/client created $ kubectl -n test get deployments.apps NAME READY UP-TO-DATE AVAILABLE AGE client 0/1 0 0 38s $ kubectl -n test describe deployments.apps client Name: client Namespace: test CreationTimestamp: Wed, 09 Feb 2022 11:21:25 +0800 Labels: app=client Annotations: deployment.kubernetes.io/revision: 1 Selector: app=client Replicas: 1 desired | 0 updated | 0 total | 0 available | 1 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 25% max unavailable, 25% max surge Pod Template: Labels: app=client Containers: busybox: Image: busybox:1.24.1 Port: <none> Host Port: <none> Args: sh -c sleep 3600 Requests: cpu: 50m memory: 2Gi Environment: <none> Mounts: <none> Volumes: <none> Conditions: Type Status Reason ---- ------ ------ Progressing True NewReplicaSetCreated Available False MinimumReplicasUnavailable ReplicaFailure True FailedCreate OldReplicaSets: <none> NewReplicaSet: client-9d57dfdf6 (0/1 replicas created) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ScalingReplicaSet 20s deployment-controller Scaled up replica set client-9d57dfdf6 to 1 $ kubectl -n test describe rs client-9d57dfdf6 Name: client-9d57dfdf6 Namespace: test Selector: app=client,pod-template-hash=9d57dfdf6 Labels: app=client pod-template-hash=9d57dfdf6 Annotations: deployment.kubernetes.io/desired-replicas: 1 deployment.kubernetes.io/max-replicas: 2 deployment.kubernetes.io/revision: 1 Controlled By: Deployment/client Replicas: 0 current / 1 desired Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed Pod Template: Labels: app=client pod-template-hash=9d57dfdf6 Containers: busybox: Image: busybox:1.24.1 Port: <none> Host Port: <none> Args: sh -c sleep 3600 Requests: cpu: 50m memory: 2Gi Environment: <none> Mounts: <none> Volumes: <none> Conditions: Type Status Reason ---- ------ ------ ReplicaFailure True FailedCreate Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-9x74f" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-62kxp" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-nx9bs" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 29s replicaset-controller Error creating: pods "client-9d57dfdf6-zs9jl" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-llrdj" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-25qrk" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-2tlxl" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 28s replicaset-controller Error creating: pods "client-9d57dfdf6-fdl4j" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 27s replicaset-controller Error creating: pods "client-9d57dfdf6-hjfnf" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=1Gi, limited: requests.memory=1Gi Warning FailedCreate 8s (x4 over 26s) replicaset-controller (combined from similar events): Error creating: pods "client-9d57dfdf6-5xkj7" is forbidden: exceeded quota: quota, requested: requests.memory=2Gi, used: requests.memory=0, limited: requests.memory=1Gi ``` ### 驗證pod數量創建一個5副本的deployment清單文件 ```yaml apiVersion: apps/v1 kind: Deployment metadata: labels: app: web name: web spec: replicas: 5 selector: matchLabels: app: web strategy: {} template: metadata: labels: app: web spec: containers: - image: nginx name: nginx resources: requests: memory: 100m cpu: 50m ``` 創建容器 ```shell $ kubectl apply -f web.yml -n test deployment.apps/web created $ kubectl -n test get pod NAME READY STATUS RESTARTS AGE web-584b96b57-24pk5 1/1 Running 0 43s web-584b96b57-czr6q 1/1 Running 0 43s web-584b96b57-m9hkv 1/1 Running 0 43s web-584b96b57-szg9x 1/1 Running 0 43s $ kubectl -n test get deployments.apps NAME READY UP-TO-DATE AVAILABLE AGE web 4/5 4 4 30s $ kubectl -n test describe rs web-584b96b57 Name: web-584b96b57 Namespace: test Selector: app=web,pod-template-hash=584b96b57 Labels: app=web pod-template-hash=584b96b57 Annotations: deployment.kubernetes.io/desired-replicas: 5 deployment.kubernetes.io/max-replicas: 7 deployment.kubernetes.io/revision: 1 Controlled By: Deployment/web Replicas: 4 current / 5 desired Pods Status: 4 Running / 0 Waiting / 0 Succeeded / 0 Failed Pod Template: Labels: app=web pod-template-hash=584b96b57 Containers: nginx: Image: nginx Port: <none> Host Port: <none> Requests: cpu: 50m memory: 100m Environment: <none> Mounts: <none> Volumes: <none> Conditions: Type Status Reason ---- ------ ------ ReplicaFailure True FailedCreate Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-m9hkv Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-24pk5 Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-szg9x Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-4ttxz" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Normal SuccessfulCreate 70s replicaset-controller Created pod: web-584b96b57-czr6q Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-jv9mp" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-7vsjh" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-7pbxc" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-sdlgw" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-ksjzx" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-gqk28" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-spczj" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 70s replicaset-controller Error creating: pods "web-584b96b57-8kzvt" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 Warning FailedCreate 20s (x12 over 69s) replicaset-controller (combined from similar events): Error creating: pods "web-584b96b57-rjkxh" is forbidden: exceeded quota: quota, requested: pods=1, used: pods=4, limited: pods=4 ``` ### 其他驗證請自行測試，這里就不進行一一驗證。 > 說明： > 1、創建deployment必須設置 `requests.memory` 和 `requests.cpu`，否則創建pod失敗 > 2、查看deployment創建成功，但是pod沒有創建出來。那得看 `rs` 創建的情況 ## 參考文章 https://kubernetes.io/zh/docs/concepts/policy/resource-quotas/