Reference architecture: up to 3,000 users · Gitlab 中文文檔

# Reference architecture: up to 3,000 users > 原文：[https://docs.gitlab.com/ee/administration/reference_architectures/3k_users.html](https://docs.gitlab.com/ee/administration/reference_architectures/3k_users.html) * [Setup components](#setup-components) * [Configure the external load balancer](#configure-the-external-load-balancer) * [Application node terminates SSL](#application-node-terminates-ssl) * [Load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl) * [Load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl) * [Ports](#ports) * [Alternate SSH Port](#alternate-ssh-port) * [Configure Redis](#configure-redis) * [Provide your own Redis instance](#provide-your-own-redis-instance) * [Standalone Redis using Omnibus GitLab](#standalone-redis-using-omnibus-gitlab) * [Configuring the primary Redis instance](#configuring-the-primary-redis-instance) * [Configuring the replica Redis instances](#configuring-the-replica-redis-instances) * [Configure Consul and Sentinel](#configure-consul-and-sentinel) * [Configure PostgreSQL](#configure-postgresql) * [Provide your own PostgreSQL instance](#provide-your-own-postgresql-instance) * [Standalone PostgreSQL using Omnibus GitLab](#standalone-postgresql-using-omnibus-gitlab) * [PostgreSQL primary node](#postgresql-primary-node) * [PostgreSQL secondary nodes](#postgresql-secondary-nodes) * [PostgreSQL post-configuration](#postgresql-post-configuration) * [Configure PgBouncer](#configure-pgbouncer) * [Configure the internal load balancer](#configure-the-internal-load-balancer) * [Configure Gitaly](#configure-gitaly) * [Gitaly TLS support](#gitaly-tls-support) * [Configure Sidekiq](#configure-sidekiq) * [Configure GitLab Rails](#configure-gitlab-rails) * [GitLab Rails post-configuration](#gitlab-rails-post-configuration) * [Configure Prometheus](#configure-prometheus) * [Configure the object storage](#configure-the-object-storage) * [Configure NFS (optional)](#configure-nfs-optional) * [Troubleshooting](#troubleshooting) # Reference architecture: up to 3,000 users[](#reference-architecture-up-to-3000-users "Permalink") 此頁面描述了最多 3,000 個用戶的 GitLab 參考架構. 有關參考架構的完整列表，請參見[可用參考架構](index.html#available-reference-architectures) . **注意：**下面記錄的 3,000 個用戶參考體系結構旨在幫助您的組織實現高度可用的 GitLab 部署. 如果您沒有專業知識或需要維護高度可用的環境，則可以遵循[2,000 個用戶的參考體系結構](2k_users.html) ，從而擁有一個更簡單且成本更低的操作環境. > * **支持的用戶（大約）：** 3,000 > * **高可用性：** True > * **測試 RPS 速率：** API：60 RPS，網站：6 RPS，Git：6 RPS | Service | Nodes | Configuration | GCP | AWS | Azure | | --- | --- | --- | --- | --- | --- | | 外部負載平衡節點 | 1 | 2 vCPU, 1.8GB Memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | | Redis | 3 | 2 個 vCPU，7.5GB 內存 | `n1-standard-2` | `m5.large` | `D2s v3` | | 領事+前哨 | 3 | 2 個 vCPU，1.8GB 內存 | `n1-highcpu-2` | `c5.large` | `F2s v2` | | PostgreSQL | 3 | 2 個 vCPU，7.5GB 內存 | `n1-standard-2` | `m5.large` | `D2s v3` | | PgBouncer | 3 | 2 個 vCPU，1.8GB 內存 | `n1-highcpu-2` | `c5.large` | `F2s v2` | | 內部負載平衡節點 | 1 | 2 vCPU, 1.8GB Memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | | Gitaly | 最少 2 個 | 4 個 vCPU，15GB 內存 | `n1-standard-4` | `m5.xlarge` | `D4s v3` | | Sidekiq | 4 | 2 個 vCPU，7.5GB 內存 | `n1-standard-2` | `m5.large` | `D2s v3` | | 亞搏體育 app Rails | 3 | 8 個 vCPU，7.2GB 內存 | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | | 監控節點 | 1 | 2 vCPU, 1.8GB Memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | | 對象存儲 | n/a | n/a | n/a | n/a | n/a | | NFS 服務器（可選，不推薦） | 1 | 4 個 vCPU，3.6GB 內存 | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | 這些架構是使用 GCP 上的[Intel Xeon E5 v3（Haswell）](https://cloud.google.com/compute/docs/cpu-platforms) CPU 平臺構建和測試的. 在不同的硬件上，您可能會發現需要對 CPU 或節點數進行相應的調整，無論是較低還是較高. 有關更多信息，請在[此處](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)找到 CPU 的[Sysbench](https://github.com/akopytov/sysbench)基準. 對于 LFS，Uploads，Artifacts 等數據對象，由于性能和可用性更好，建議在 NFS 上盡可能使用[對象存儲服務](#configure-the-object-storage) . 由于這不需要設置節點，因此在上表中將其標記為不適用（n / a）. ## Setup components[](#setup-components "Permalink") 設置 GitLab 及其組件以容納多達 3,000 個用戶： 1. [配置外部負載平衡節點](#configure-the-external-load-balancer) ，該[節點](#configure-the-external-load-balancer)將處理兩個 GitLab 應用程序服務節點的負載平衡. 2. [Configure Redis](#configure-redis). 3. [Configure Consul and Sentinel](#configure-consul-and-sentinel). 4. [配置 PostgreSQL](#configure-postgresql) （GitLab 的數據庫）. 5. [Configure PgBouncer](#configure-pgbouncer). 6. [Configure the internal load balancing node](#configure-the-internal-load-balancer) 7. [配置 Gitaly](#configure-gitaly) ，它提供對 Git 存儲庫的訪問. 8. [Configure Sidekiq](#configure-sidekiq). 9. [配置主 GitLab Rails 應用程序](#configure-gitlab-rails)以運行 Puma / Unicorn，Workhorse，GitLab Shell，并服務所有前端請求（UI，API，基于 HTTP / SSH 的 Git）. 10. [配置 Prometheus](#configure-prometheus)來監視您的 GitLab 環境. 11. [配置](#configure-the-object-storage)用于共享數據對象[的對象存儲](#configure-the-object-storage) . 12. [將 NFS（可選）配置](#configure-nfs-optional)為具有共享磁盤存儲服務，以替代 Gitaly 和/或對象存儲（盡管不建議這樣做）. GitLab 頁面需要 NFS，如果不使用該功能，則可以跳過此步驟. 我們從同一 10.6.0.0/16 專用網絡范圍內的所有服務器開始，它們可以在這些地址上自由地相互連接. 這是每臺機器和分配的 IP 的列表和說明： * `10.6.0.10` ：外部負載平衡器 * `10.6.0.61`主要 * `10.6.0.62` ：返回副本 1 * `10.6.0.63` ：返回副本 2 * `10.6.0.11` ：領事/前哨 1 * `10.6.0.12` ：領事/前哨 2 * `10.6.0.13` ：領事/前哨 3 * `10.6.0.31`主 * `10.6.0.32`中學 1 * `10.6.0.33`中學 2 * `10.6.0.21` ：PgBouncer 1 * `10.6.0.22` ：PgBouncer 2 * `10.6.0.23` ：PgBouncer 3 * `10.6.0.20` ：內部負載均衡器 * `10.6.0.51` ：Gitaly 1 * `10.6.0.52` ：Gitaly 2 * `10.6.0.71` ：Sidekiq 1 * `10.6.0.72` ：Sidekiq 2 * `10.6.0.73` ：Sidekiq 3 * `10.6.0.74` ：Sidekiq 4 * `10.6.0.41`應用程序 1 * `10.6.0.42`應用程序 2 * `10.6.0.43`應用程序 3 * `10.6.0.81` ：普羅米修斯 ## Configure the external load balancer[](#configure-the-external-load-balancer "Permalink") **注意：**此體系結構已通過[HAProxy](https://www.haproxy.org/)作為負載均衡器進行了測試和驗證. 盡管也可以使用具有類似功能集的其他負載均衡器，但這些負載均衡器尚未經過驗證. 在主動/主動 GitLab 配置中，您將需要一個負載均衡器來將流量路由到應用程序服務器. 有關使用負載均衡器或進行確切配置的細節超出了 GitLab 文檔的范圍. 我們希望，如果您要管理像 GitLab 這樣的多節點系統，那么已經選擇了負載均衡器. 一些示例包括 HAProxy（開源），F5 Big-IP LTM 和 Citrix Net Scaler. 本文檔將概述需要在 GitLab 上使用哪些端口和協議. 下一個問題是如何在環境中處理 SSL. 有幾種不同的選擇： * [The application node terminates SSL](#application-node-terminates-ssl). * [負載平衡器終止沒有后端 SSL 的 SSL，](#load-balancer-terminates-ssl-without-backend-ssl)并且負載平衡器與應用程序節點之間的通信不安全. * [負載均衡器使用后端 SSL 終止 SSL，](#load-balancer-terminates-ssl-with-backend-ssl)并且負載均衡器與應用程序節點之間的通信是*安全*的. ### Application node terminates SSL[](#application-node-terminates-ssl "Permalink") 配置您的負載均衡器以將端口 443 上的連接作為`TCP`而不是`HTTP(S)`協議進行傳遞. 這會將連接直接傳遞到應用程序節點的 NGINX 服務. NGINX 將具有 SSL 證書并在端口 443 上偵聽. 有關管理 SSL 證書和配置 NGINX 的詳細信息，請參見[NGINX HTTPS 文檔](https://docs.gitlab.com/omnibus/settings/nginx.html) . ### Load balancer terminates SSL without backend SSL[](#load-balancer-terminates-ssl-without-backend-ssl "Permalink") 將您的負載均衡器配置為使用`HTTP(S)`協議而不是`TCP` . 然后，負載平衡器將負責管理 SSL 證書和終止 SSL. 由于負載均衡器和 GitLab 之間的通信將不安全，因此需要一些其他配置. 有關詳細信息，請參見[NGINX 代理的 SSL 文檔](https://docs.gitlab.com/omnibus/settings/nginx.html) . ### Load balancer terminates SSL with backend SSL[](#load-balancer-terminates-ssl-with-backend-ssl "Permalink") Configure your load balancer(s) to use the ‘HTTP(S)’ protocol rather than ‘TCP’. The load balancer(s) will be responsible for managing SSL certificates that end users will see. 在這種情況下，負載均衡器和 NGINX 之間的流量也將是安全的. 無需為代理 SSL 添加配置，因為連接將一直保持安全. 但是，需要將配置添加到 GitLab 來配置 SSL 證書. 有關管理 SSL 證書和配置 NGINX 的詳細信息，請參見[NGINX HTTPS 文檔](https://docs.gitlab.com/omnibus/settings/nginx.html) . ### Ports[](#ports "Permalink") 下表顯示了要使用的基本端口. | LB 端口 | 后端端口 | Protocol | | --- | --- | --- | | 80 | 80 | HTTP（ *1* ） | | 443 | 443 | TCP 或 HTTPS（ *1* ）（ *2* ） | | 22 | 22 | TCP | * （ *1* ）： [Web 終端](../../ci/environments/index.html#web-terminals)支持要求您的負載平衡器正確處理 WebSocket 連接. 當使用 HTTP 或 HTTPS 代理，這意味著負載平衡器必須被配置為通過`Connection`和`Upgrade`逐跳頭. 有關更多詳細信息，請參見[Web 終端](../integration/terminal.html)集成指南. * （ *2* ）：當對端口 443 使用 HTTPS 協議時，需要向負載均衡器添加 SSL 證書. 如果您想在 GitLab 應用程序服務器上終止 SSL，請使用 TCP 協議. 如果您使用具有自定義域支持的 GitLab 頁面，則將需要一些其他端口配置. GitLab 頁面需要一個單獨的虛擬 IP 地址. 配置 DNS，將`pages_external_url`的`/etc/gitlab/gitlab.rb`指向新的虛擬 IP 地址. 有關更多信息，請參見[GitLab 頁面文檔](../pages/index.html) . | LB 端口 | 后端端口 | Protocol | | --- | --- | --- | | 80 | 變化（ *1* ） | HTTP | | 443 | 變化（ *1* ） | TCP（ *2* ） | * （ *1* ）：GitLab 頁面的后端端口取決于`gitlab_pages['external_http']`和`gitlab_pages['external_https']`設置. 有關更多詳細信息，請參見[GitLab Pages 文檔](../pages/index.html) . * （ *2* ）：GitLab 頁面的端口 443 應該始終使用 TCP 協議. 用戶可以使用自定義 SSL 配置自定義域，如果 SSL 在負載均衡器處終止，則不可能. #### Alternate SSH Port[](#alternate-ssh-port "Permalink") 某些組織有禁止打開 SSH 端口 22 的策略.在這種情況下，配置允許用戶在端口 443 上使用 SSH 的備用 SSH 主機名可能會有所幫助.與其他 GitLab 相比，備用 SSH 主機名將需要一個新的虛擬 IP 地址.上面的 HTTP 配置. 為備用 SSH 主機名（例如`altssh.gitlab.example.com`配置 DNS. | LB 端口 | 后端端口 | Protocol | | --- | --- | --- | | 443 | 22 | TCP | [Back to setup components](#setup-components) ## Configure Redis[](#configure-redis "Permalink") 使用[Redis 的](https://s0redis0io.icopy.site/)可擴展環境，可以使用**一次** X **副本**拓撲與[Redis 的哨兵](https://s0redis0io.icopy.site/topics/sentinel)服務來觀看，并自動啟動故障轉移過程. 如果與 Sentinel 一起使用，Redis 需要身份驗證. 有關更多信息，請參見[Redis 安全性](https://s0redis0io.icopy.site/topics/security)文檔. 我們建議結合使用 Redis 密碼和嚴格的防火墻規則來保護您的 Redis 服務. 強烈建議您在使用 GitLab 配置 Redis 之前閱讀[Redis Sentinel](https://s0redis0io.icopy.site/topics/sentinel)文檔，以充分了解拓撲和體系結構. 在本節中，將指導您配置與 GitLab 一起使用的外部 Redis 實例. 以下 IP 將作為示例： * `10.6.0.61`主要 * `10.6.0.62` ：返回副本 1 * `10.6.0.63` ：返回副本 2 ### Provide your own Redis instance[](#provide-your-own-redis-instance "Permalink") 來自云提供商（例如 AWS ElastiCache）的托管 Redis 將可以使用. 如果這些服務支持高可用性，請確保它**不是** Redis 群集類型. 需要 Redis 5.0 或更高版本，因為這是從 GitLab 13.0 開始的 Omnibus GitLab 軟件包附帶的版本. 較舊的 Redis 版本不支持 SPOP 的可選 count 參數，這對于[合并火車](../../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.html)現在是必需的. 注意 Redis 節點的 IP 地址或主機名，端口和密碼（如果需要）. 這些在以后配置[GitLab 應用程序服務器](#configure-gitlab-rails)時是必需的. ### Standalone Redis using Omnibus GitLab[](#standalone-redis-using-omnibus-gitlab "Permalink") 這是我們安裝和設置新 Redis 實例的部分. Redis 設置的要求如下： 1. 所有 Redis 節點必須能夠互相通信并接受通過 Redis（ `6379` ）和 Sentinel（ `26379` ）端口的傳入連接（除非您更改默認端口）. 2. 托管 GitLab 應用程序的服務器必須能夠訪問 Redis 節點. 3. 使用防火墻保護節點免受來自外部網絡（ [Internet](https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png) ）的訪問. **注意：** Redis 節點（主節點和副本節點）將需要使用`redis['password']`定義的相同密碼. 在故障轉移期間的任何時間，Sentinels 都可以重新配置節點并將其狀態從主節點更改為副本節點，反之亦然. #### Configuring the primary Redis instance[](#configuring-the-primary-redis-instance "Permalink") 1. SSH 進入**主** Redis 服務器. 2. 從 GitLab 下載頁面使用**步驟 1 和 2** [下載/安裝](https://about.gitlab.com/install/)所需的 Omnibus GitLab 軟件包. * 確保選擇正確的 Omnibus 軟件包，并使用與當前安裝相同的版本和類型（社區版，企業版）. * 不要完成下載頁面上的任何其他步驟. 3. 編輯`/etc/gitlab/gitlab.rb`并添加內容： ``` # Specify server role as 'redis_master_role' roles ['redis_master_role'] # IP address pointing to a local IP that the other machines can reach to. # You can also set bind to '0.0.0.0' which listen in all interfaces. # If you really need to bind to an external accessible IP, make # sure you add extra firewall rules to prevent unauthorized access. redis['bind'] = '10.6.0.61' # Define a port so Redis can listen for TCP requests which will allow other # machines to connect to it. redis['port'] = 6379 # Set up password authentication for Redis (use the same password in all nodes). redis['password'] = 'redis-password-goes-here' ## Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } # Set the network addresses that the exporters will listen on node_exporter['listen_address'] = '0.0.0.0:9100' redis_exporter['listen_address'] = '0.0.0.0:9121' redis_exporter['flags'] = { 'redis.addr' => 'redis://10.6.0.61:6379', 'redis.password' => 'redis-password-goes-here', } # Disable auto migrations gitlab_rails['auto_migrate'] = false ``` 4. [重新配置 Omnibus GitLab，](../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效. **注意：**您可以將多個角色（如哨兵和 Redis）指定為： `roles ['redis_sentinel_role', 'redis_master_role']` . 閱讀有關[角色的](https://docs.gitlab.com/omnibus/roles/)更多信息. 您可以通過以下方式列出當前 Redis 主副本服務器狀態： ``` /opt/gitlab/embedded/bin/redis-cli -h <host> -a 'redis-password-goes-here' info replication ``` 通過以下方式顯示正在運行的 GitLab 服務： ``` gitlab-ctl status ``` 輸出應類似于以下內容： ``` run: consul: (pid 30043) 76863s; run: log: (pid 29691) 76892s run: logrotate: (pid 31152) 3070s; run: log: (pid 29595) 76908s run: node-exporter: (pid 30064) 76862s; run: log: (pid 29624) 76904s run: redis: (pid 30070) 76861s; run: log: (pid 29573) 76914s run: redis-exporter: (pid 30075) 76861s; run: log: (pid 29674) 76896s ``` #### Configuring the replica Redis instances[](#configuring-the-replica-redis-instances "Permalink") 1. SSH 進入**副本** Redis 服務器. 2. 從 GitLab 下載頁面使用**步驟 1 和 2** [下載/安裝](https://about.gitlab.com/install/)所需的 Omnibus GitLab 軟件包. * 確保選擇正確的 Omnibus 軟件包，并使用與當前安裝相同的版本和類型（社區版，企業版）. * 不要完成下載頁面上的任何其他步驟. 3. 編輯`/etc/gitlab/gitlab.rb`并添加內容： ``` # Specify server role as 'redis_replica_role' roles ['redis_replica_role'] # IP address pointing to a local IP that the other machines can reach to. # You can also set bind to '0.0.0.0' which listen in all interfaces. # If you really need to bind to an external accessible IP, make # sure you add extra firewall rules to prevent unauthorized access. redis['bind'] = '10.6.0.62' # Define a port so Redis can listen for TCP requests which will allow other # machines to connect to it. redis['port'] = 6379 # The same password for Redis authentication you set up for the primary node. redis['password'] = 'redis-password-goes-here' # The IP of the primary Redis node. redis['master_ip'] = '10.6.0.61' # Port of primary Redis server, uncomment to change to non default. Defaults # to `6379`. #redis['master_port'] = 6379 ## Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } # Set the network addresses that the exporters will listen on node_exporter['listen_address'] = '0.0.0.0:9100' redis_exporter['listen_address'] = '0.0.0.0:9121' redis_exporter['flags'] = { 'redis.addr' => 'redis://10.6.0.62:6379', 'redis.password' => 'redis-password-goes-here', } # Disable auto migrations gitlab_rails['auto_migrate'] = false ``` 4. [重新配置 Omnibus GitLab，](../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效. 5. 對于所有其他副本節點，請再次執行該步驟，并確保正確設置 IP. **注意：**您可以將多個角色（如哨兵和 Redis）指定為： `roles ['redis_sentinel_role', 'redis_master_role']` . 閱讀有關[角色的](https://docs.gitlab.com/omnibus/roles/)更多信息. 故障轉移后， `/etc/gitlab/gitlab.rb`在`/etc/gitlab/gitlab.rb`再次更改這些值，因為節點將由[Sentinels](#configure-consul-and-sentinel)管理，即使在`gitlab-ctl reconfigure` ，它們也將通過恢復配置恢復.同樣的哨兵支持高級[配置選項](https://docs.gitlab.com/omnibus/settings/redis.html) ，可以根據需要添加. [Back to setup components](#setup-components) ## Configure Consul and Sentinel[](#configure-consul-and-sentinel "Permalink") **注意：**如果您使用的是外部 Redis Sentinel 實例，請確保從 Sentinel 配置中排除`requirepass`參數. 此參數將導致客戶端報告`NOAUTH Authentication required.` . [Redis Sentinel 3.2.x 不支持密碼身份驗證](https://github.com/antirez/redis/issues/3279) . 現在已經全部安裝了 Redis 服務器，讓我們配置 Sentinel 服務器. 以下 IP 將作為示例： * `10.6.0.11` ：領事/前哨 1 * `10.6.0.12` ：領事/前哨 2 * `10.6.0.13` ：領事/前哨 3 要配置 Sentinel： 1. SSH 進入將托管 Consul / Sentinel 的服務器. 2. 從 GitLab 下載頁面使用**步驟 1 和 2** [下載/安裝](https://about.gitlab.com/install/) Omnibus GitLab 企業版軟件包. * 確保選擇正確的 Omnibus 軟件包，并且與 GitLab 應用程序正在運行的版本相同. * 不要完成下載頁面上的任何其他步驟. 3. 編輯`/etc/gitlab/gitlab.rb`并添加內容： ``` roles ['redis_sentinel_role', 'consul_role'] # Must be the same in every sentinel node redis['master_name'] = 'gitlab-redis' # The same password for Redis authentication you set up for the primary node. redis['master_password'] = 'redis-password-goes-here' # The IP of the primary Redis node. redis['master_ip'] = '10.6.0.61' # Define a port so Redis can listen for TCP requests which will allow other # machines to connect to it. redis['port'] = 6379 # Port of primary Redis server, uncomment to change to non default. Defaults # to `6379`. #redis['master_port'] = 6379 ## Configure Sentinel sentinel['bind'] = '10.6.0.11' # Port that Sentinel listens on, uncomment to change to non default. Defaults # to `26379`. # sentinel['port'] = 26379 ## Quorum must reflect the amount of voting sentinels it take to start a failover. ## Value must NOT be greater then the amount of sentinels. ## ## The quorum can be used to tune Sentinel in two ways: ## 1\. If a the quorum is set to a value smaller than the majority of Sentinels ## we deploy, we are basically making Sentinel more sensible to primary failures, ## triggering a failover as soon as even just a minority of Sentinels is no longer ## able to talk with the primary. ## 1\. If a quorum is set to a value greater than the majority of Sentinels, we are ## making Sentinel able to failover only when there are a very large number (larger ## than majority) of well connected Sentinels which agree about the primary being down.s sentinel['quorum'] = 2 ## Consider unresponsive server down after x amount of ms. # sentinel['down_after_milliseconds'] = 10000 ## Specifies the failover timeout in milliseconds. It is used in many ways: ## ## - The time needed to re-start a failover after a previous failover was ## already tried against the same primary by a given Sentinel, is two ## times the failover timeout. ## ## - The time needed for a replica replicating to a wrong primary according ## to a Sentinel current configuration, to be forced to replicate ## with the right primary, is exactly the failover timeout (counting since ## the moment a Sentinel detected the misconfiguration). ## ## - The time needed to cancel a failover that is already in progress but ## did not produced any configuration change (REPLICAOF NO ONE yet not ## acknowledged by the promoted replica). ## ## - The maximum time a failover in progress waits for all the replica to be ## reconfigured as replicas of the new primary. However even after this time ## the replicas will be reconfigured by the Sentinels anyway, but not with ## the exact parallel-syncs progression as specified. # sentinel['failover_timeout'] = 60000 ## Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { server: true, retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } # Set the network addresses that the exporters will listen on node_exporter['listen_address'] = '0.0.0.0:9100' redis_exporter['listen_address'] = '0.0.0.0:9121' # Disable auto migrations gitlab_rails['auto_migrate'] = false ``` 4. [重新配置 Omnibus GitLab，](../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效. 5. 對于其他所有 Consul / Sentinel 節點，請再次執行步驟，并確保設置了正確的 IP. **注意：**第三個 Consul 服務器的配置完成后，將選舉 Consul 負責人. 查看領事日志`sudo gitlab-ctl tail consul`將顯示`...[INFO] consul: New leader elected: ...` You can list the current Consul members (server, client): ``` sudo /opt/gitlab/embedded/bin/consul members ``` 您可以驗證 GitLab 服務正在運行： ``` sudo gitlab-ctl status ``` 輸出應類似于以下內容： ``` run: consul: (pid 30074) 76834s; run: log: (pid 29740) 76844s run: logrotate: (pid 30925) 3041s; run: log: (pid 29649) 76861s run: node-exporter: (pid 30093) 76833s; run: log: (pid 29663) 76855s run: sentinel: (pid 30098) 76832s; run: log: (pid 29704) 76850s ``` [Back to setup components](#setup-components) ## Configure PostgreSQL[](#configure-postgresql "Permalink") 在本節中，將指導您配置與 GitLab 一起使用的外部 PostgreSQL 數據庫. ### Provide your own PostgreSQL instance[](#provide-your-own-postgresql-instance "Permalink") 如果您將 GitLab 托管在云提供商上，則可以選擇將托管服務用于 PostgreSQL. 例如，AWS 提供了運行 PostgreSQL 的托管關系數據庫服務（RDS）. 如果您使用云托管服務，或提供自己的 PostgreSQL： 1. 根據[數據庫要求文檔](../../install/requirements.html#database)設置 PostgreSQL. 2. 使用您選擇的密碼設置一個`gitlab`用戶名. `gitlab`用戶需要特權才能創建`gitlabhq_production`數據庫. 3. 使用適當的詳細信息配置 GitLab 應用程序服務器. [配置 GitLab Rails 應用程序](#configure-gitlab-rails)涵蓋了此步驟. ### Standalone PostgreSQL using Omnibus GitLab[](#standalone-postgresql-using-omnibus-gitlab "Permalink") 以下 IP 將作為示例： * `10.6.0.31`主 * `10.6.0.32`中學 1 * `10.6.0.33`中學 2 首先，請確保**在每個節點上** [安裝](https://about.gitlab.com/install/) Linux GitLab 軟件包. 按照以下步驟，從步驟 1 安裝必需的依賴項，并從步驟 2 添加 GitLab 軟件包存儲庫.在第二步中安裝 GitLab 時，請勿提供`EXTERNAL_URL`值. #### PostgreSQL primary node[](#postgresql-primary-node "Permalink") 1. SSH 進入 PostgreSQL 主節點. 2. 為 PostgreSQL 用戶名/密碼對生成密碼哈希. 假設您將使用默認用戶名`gitlab` （推薦）. 該命令將要求輸入密碼和確認. 將此命令在下一步中輸出的值用作`<postgresql_password_hash>`的值： ``` sudo gitlab-ctl pg-password-md5 gitlab ``` 3. 為 PgBouncer 用戶名/密碼對生成密碼哈希. 假設您將使用`pgbouncer`的默認用戶名（推薦）. 該命令將要求輸入密碼和確認. 將此命令在下一步中輸出的值用作`<pgbouncer_password_hash>`的值： ``` sudo gitlab-ctl pg-password-md5 pgbouncer ``` 4. 為 Consul 數據庫用戶名/密碼對生成密碼哈希. 假設您將使用默認用戶名`gitlab-consul` （推薦）. 該命令將要求輸入密碼和確認. 將此命令在下一步中輸出的值用作`<consul_password_hash>`的值： ``` sudo gitlab-ctl pg-password-md5 gitlab-consul ``` 5. 在主數據庫節點上，編輯`/etc/gitlab/gitlab.rb`替換`/etc/gitlab/gitlab.rb` `# START user configuration`部分中記錄的值： ``` # Disable all components except PostgreSQL and Repmgr and Consul roles ['postgres_role'] # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' postgresql['hot_standby'] = 'on' postgresql['wal_level'] = 'replica' postgresql['shared_preload_libraries'] = 'repmgr_funcs' # Disable automatic database migrations gitlab_rails['auto_migrate'] = false # Configure the Consul agent consul['services'] = %w(postgresql) # START user configuration # Please set the real values as explained in Required Information section # # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = '<postgresql_password_hash>' # Set `max_wal_senders` to one more than the number of database nodes in the cluster. # This is used to prevent replication from using up all of the # available database connections. postgresql['max_wal_senders'] = 4 postgresql['max_replication_slots'] = 4 # Replace XXX.XXX.XXX.XXX/YY with Network Address postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) ## Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters will listen on for monitoring node_exporter['listen_address'] = '0.0.0.0:9100' postgres_exporter['listen_address'] = '0.0.0.0:9187' postgres_exporter['dbname'] = 'gitlabhq_production' postgres_exporter['password'] = '<postgresql_password_hash>' ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } # # END user configuration ``` 6. [重新配置 GitLab，](../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效. 7. 您可以通過以下方式列出當前 PostgreSQL 主，輔助節點的狀態： ``` sudo /opt/gitlab/bin/gitlab-ctl repmgr cluster show ``` 8. 驗證 GitLab 服務正在運行： ``` sudo gitlab-ctl status ``` 輸出應類似于以下內容： ``` run: consul: (pid 30593) 77133s; run: log: (pid 29912) 77156s run: logrotate: (pid 23449) 3341s; run: log: (pid 29794) 77175s run: node-exporter: (pid 30613) 77133s; run: log: (pid 29824) 77170s run: postgres-exporter: (pid 30620) 77132s; run: log: (pid 29894) 77163s run: postgresql: (pid 30630) 77132s; run: log: (pid 29618) 77181s run: repmgrd: (pid 30639) 77132s; run: log: (pid 29985) 77150s ``` [Back to setup components](#setup-components) #### PostgreSQL secondary nodes[](#postgresql-secondary-nodes "Permalink") 1. 在兩個輔助節點上，添加與上面為主要節點指定的配置相同的附加設置，該設置將告知`gitlab-ctl`最初它們是備用節點，無需嘗試將它們注冊為主要節點： ``` # Disable all components except PostgreSQL and Repmgr and Consul roles ['postgres_role'] # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' postgresql['hot_standby'] = 'on' postgresql['wal_level'] = 'replica' postgresql['shared_preload_libraries'] = 'repmgr_funcs' # Disable automatic database migrations gitlab_rails['auto_migrate'] = false # Configure the Consul agent consul['services'] = %w(postgresql) # Specify if a node should attempt to be primary on initialization. repmgr['master_on_initialization'] = false # START user configuration # Please set the real values as explained in Required Information section # # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = '<postgresql_password_hash>' # Set `max_wal_senders` to one more than the number of database nodes in the cluster. # This is used to prevent replication from using up all of the # available database connections. postgresql['max_wal_senders'] = 4 postgresql['max_replication_slots'] = 4 # Replace XXX.XXX.XXX.XXX/YY with Network Address postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) ## Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters will listen on for monitoring node_exporter['listen_address'] = '0.0.0.0:9100' postgres_exporter['listen_address'] = '0.0.0.0:9187' postgres_exporter['dbname'] = 'gitlabhq_production' postgres_exporter['password'] = '<postgresql_password_hash>' ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } # END user configuration ``` 2. [重新配置 GitLab，](../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效. 支持高級[配置選項](https://docs.gitlab.com/omnibus/settings/database.html) ，可以根據需要添加. [Back to setup components](#setup-components) #### PostgreSQL post-configuration[](#postgresql-post-configuration "Permalink") SSH 進入**主節點** ： 1. 打開數據庫提示： ``` gitlab-psql -d gitlabhq_production ``` 2. Enable the `pg_trgm` extension: ``` CREATE EXTENSION pg_trgm; ``` 3. 鍵入`\q`并按 Enter 退出數據庫提示. 4. 驗證集群是否已用一個節點初始化： ``` gitlab-ctl repmgr cluster show ``` 輸出應類似于以下內容： ``` Role | Name | Upstream | Connection String ----------+----------|----------|---------------------------------------- * master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr ``` 5. 在連接字符串中記下主機名或 IP 地址： `host=HOSTNAME` . 在下一節中，我們將主機名稱為`<primary_node_name>` . 如果該值不是 IP 地址，則必須是可解析的名稱（通過 DNS 或`/etc/hosts` ） SSH 進入**輔助節點** ： 1. 設置 repmgr 備用數據庫： ``` gitlab-ctl repmgr standby setup <primary_node_name> ``` Do note that this will remove the existing data on the node. The command has a wait time. 輸出應類似于以下內容： ``` Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data If this is not what you want, hit Ctrl-C now to exit To skip waiting, rerun with the -w option Sleeping for 30 seconds Stopping the database Removing the data Cloning the data Starting the database Registering the node with the cluster ok: run: repmgrd: (pid 19068) 0s ``` 在繼續之前，請確保正確配置了數據庫. 在**主**節點上運行以下命令以驗證復制是否正常工作，并且輔助節點是否出現在群集中： ``` gitlab-ctl repmgr cluster show ``` 輸出應類似于以下內容： ``` Role | Name | Upstream | Connection String ----------+---------|-----------|------------------------------------------------ * master | MASTER | | host=<primary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr ``` 如果任何節點的"角色"列顯示"失敗"，請在繼續操作之前檢查" [故障排除"部分](troubleshooting.html) . 另外，請檢查`repmgr-check-master`命令在每個節點上是否都能正常工作： ``` su - gitlab-consul gitlab-ctl repmgr-check-master || echo 'This node is a standby repmgr node' ``` 此命令依靠退出代碼來告訴 Consul 特定節點是主節點還是輔助節點. 這里最重要的是該命令不會產生錯誤. 如果有錯誤，很可能是由于`gitlab-consul`數據庫用戶權限不正確`gitlab-consul` . 在繼續之前，請檢查" [故障排除"部分](troubleshooting.html) . [Back to setup components](#setup-components) ## Configure PgBouncer[](#configure-pgbouncer "Permalink") 現在已經安裝了 PostgreSQL 服務器，讓我們配置 PgBouncer. 以下 IP 將作為示例： * `10.6.0.21` ：PgBouncer 1 * `10.6.0.22` ：PgBouncer 2 * `10.6.0.23` ：PgBouncer 3 1. 在每個 PgBouncer 節點上，編輯`/etc/gitlab/gitlab.rb` ，并將`<consul_password_hash>`和`<pgbouncer_password_hash>`替換為[之前設置](#postgresql-primary-node)的密碼哈希： ``` # Disable all components except Pgbouncer and Consul agent roles ['pgbouncer_role'] # Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) pgbouncer['users'] = { 'gitlab-consul': { password: '<consul_password_hash>' }, 'pgbouncer': { password: '<pgbouncer_password_hash>' } } # Configure Consul agent consul['watchers'] = %w(postgresql) consul['enable'] = true consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13) } # Enable service discovery for Prometheus consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters will listen on node_exporter['listen_address'] = '0.0.0.0:9100' pgbouncer_exporter['listen_address'] = '0.0.0.0:9188' ``` 2. [重新配置 Omnibus GitLab，](../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效. 3. 創建一個`.pgpass`文件，以便 Consul 能夠重新加載 PgBouncer. 詢問時兩次輸入 PgBouncer 密碼： ``` gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul ``` 4. 確保每個節點都在與當前主節點通信： ``` gitlab-ctl pgb-console # You will be prompted for PGBOUNCER_PASSWORD ``` 如果出現錯誤`psql: ERROR: Auth failed`輸入密碼后`psql: ERROR: Auth failed` ，請確保您以前以正確的格式生成了 MD5 密碼哈希. 正確的格式是連接密碼和用戶名`PASSWORDUSERNAME` . 例如， `Sup3rS3cr3tpgbouncer`將是為`pgbouncer`用戶生成 MD5 密碼哈希所需的文本. 5. 控制臺提示可用后，請運行以下查詢： ``` show databases ; show clients ; ``` 輸出應類似于以下內容： ``` name | host | port | database | force_user | pool_size | reserve_pool | pool_mode | max_connections | current_connections ---------------------+-------------+------+---------------------+------------+-----------+--------------+-----------+-----------------+--------------------- gitlabhq_production | MASTER_HOST | 5432 | gitlabhq_production | | 20 | 0 | | 0 | 0 pgbouncer | | 6432 | pgbouncer | pgbouncer | 2 | 0 | statement | 0 | 0 (2 rows) type | user | database | state | addr | port | local_addr | local_port | connect_time | request_time | ptr | link | remote_pid | tls ------+-----------+---------------------+---------+----------------+-------+------------+------------+---------------------+---------------------+-----------+------+------------+----- C | pgbouncer | pgbouncer | active | 127.0.0.1 | 56846 | 127.0.0.1 | 6432 | 2017-08-21 18:09:59 | 2017-08-21 18:10:48 | 0x22b3880 | | 0 | (2 rows) ``` 6. 驗證 GitLab 服務正在運行： ``` sudo gitlab-ctl status ``` The output should be similar to the following: ``` run: consul: (pid 31530) 77150s; run: log: (pid 31106) 77182s run: logrotate: (pid 32613) 3357s; run: log: (pid 30107) 77500s run: node-exporter: (pid 31550) 77149s; run: log: (pid 30138) 77493s run: pgbouncer: (pid 32033) 75593s; run: log: (pid 31117) 77175s run: pgbouncer-exporter: (pid 31558) 77148s; run: log: (pid 31498) 77156s ``` [Back to setup components](#setup-components) ### Configure the internal load balancer[](#configure-the-internal-load-balancer "Permalink") 如果按照建議運行多個 PgBouncer 節點，那么此時，您將需要設置一個 TCP 內部負載均衡器以正確地服務每個負載均衡器. 以下 IP 將作為示例： * `10.6.0.20` ：內部負載均衡器使用[HAProxy 的方法](https://www.haproxy.org/)如下： ``` global log /dev/log local0 log localhost local1 notice log stdout format raw local0 defaults log global default-server inter 10s fall 3 rise 2 balance leastconn frontend internal-pgbouncer-tcp-in bind *:6432 mode tcp option tcplog default_backend pgbouncer backend pgbouncer mode tcp option tcp-check server pgbouncer1 10.6.0.21:6432 check server pgbouncer2 10.6.0.22:6432 check server pgbouncer3 10.6.0.23:6432 check ``` 請參閱您首選的負載均衡器的文檔以獲取更多指導. [Back to setup components](#setup-components) ## Configure Gitaly[](#configure-gitaly "Permalink") 在自己的服務器上部署 Gitaly 可以使大于單個計算機的 GitLab 安裝受益. Gitaly 節點要求取決于客戶數據，特別是項目數量及其存儲庫大小. 建議將兩個節點作為絕對最小值. 每個 Gitaly 節點應存儲的數據不超過 5TB，并將[`gitaly-ruby`工作者](../gitaly/index.html#gitaly-ruby)的數量設置為可用 CPU 的 20％. 根據以上建議，應結合其他節點并結合對預期數據大小和分布的審查. 強烈建議所有 Gitaly 節點都安裝 SSD 磁盤，因為 Gitaly I / O 繁重，因此其讀操作的吞吐量至少為 8000 IOPS，寫操作的吞吐量至少為 2,000 IOPS. 這些 IOPS 值僅建議作為啟動器使用，因為隨著時間的推移，它們可能會根據環境工作負載的規模而調整得更高或更低. 如果您在 Cloud provider 上運行環境，則可能需要參考其文檔以了解如何正確配置 IOPS. 注意事項： * GitLab Rails 應用程序將[存儲庫分](../repository_storage_paths.html)片到[存儲庫中](../repository_storage_paths.html) . * Gitaly 服務器可以托管一個或多個存儲. * 一個 GitLab 服務器可以使用一個或多個 Gitaly 服務器. * 必須以對所有 Gitaly 客戶端正確解析的方式指定 Gitaly 地址. * Gitaly 服務器一定不能暴露在公共互聯網上，因為默認情況下，Gitaly 的網絡流量是未加密的. 強烈建議使用防火墻以限制對 Gitaly 服務器的訪問. 另一種選擇是[使用 TLS](#gitaly-tls-support) . **提示：**有關 Gitaly 歷史和網絡體系結構的更多信息，請參見[獨立的 Gitaly 文檔](../gitaly/index.html) . 注意： **注意：** Gitaly 文檔中引用的令牌只是管理員選擇的任意密碼. 它與為 GitLab API 創建的令牌或其他類似的 Web API 令牌無關. 下面我們描述如何配置兩個具有 IP 和域名的 Gitaly 服務器： * `10.6.0.51` 1（ `gitaly1.internal` ） * `10.6.0.52` 2（ `gitaly2.internal` ）假定該秘密令牌為`gitalysecret` ，并且您的 GitLab 安裝具有三個存儲庫存儲： * `default`為 Gitaly 1 * `storage1`在 Gitaly 1 * `storage2`上 Gitaly 2 在每個節點上： 1. 從 GitLab 下載頁面使用**步驟 1 和 2** [下載/安裝](https://about.gitlab.com/install/)所需的 Omnibus GitLab 軟件包，但**不**提供`EXTERNAL_URL`值. 2. 編輯`/etc/gitlab/gitlab.rb`以配置存儲路徑，啟用網絡偵聽器并配置令牌： ``` # /etc/gitlab/gitlab.rb # Gitaly and GitLab use two shared secrets for authentication, one to authenticate gRPC requests # to Gitaly, and a second for authentication callbacks from GitLab-Shell to the GitLab internal API. # The following two values must be the same as their respective values # of the GitLab Rails application setup gitaly['auth_token'] = 'gitlaysecret' gitlab_shell['secret_token'] = 'shellsecret' # Avoid running unnecessary services on the Gitaly server postgresql['enable'] = false redis['enable'] = false nginx['enable'] = false puma['enable'] = false unicorn['enable'] = false sidekiq['enable'] = false gitlab_workhorse['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false # If you run a seperate monitoring node you can disable these services alertmanager['enable'] = false prometheus['enable'] = false # Prevent database connections during 'gitlab-ctl reconfigure' gitlab_rails['rake_cache_clear'] = false gitlab_rails['auto_migrate'] = false # Configure the gitlab-shell API callback URL. Without this, `git push` will # fail. This can be your 'front door' GitLab URL or an internal load # balancer. # Don't forget to copy `/etc/gitlab/gitlab-secrets.json` from web server to Gitaly server. gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' # Make Gitaly accept connections on all network interfaces. You must use # firewalls to restrict access to this address/port. # Comment out following line if you only want to support TLS connections gitaly['listen_addr'] = "0.0.0.0:8075" ## Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters will listen on for monitoring gitaly['prometheus_listen_addr'] = "0.0.0.0:9236" node_exporter['listen_address'] = '0.0.0.0:9100' gitlab_rails['prometheus_address'] = '10.6.0.81:9090' ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } ``` 3. 對于每個服務器，將以下內容附加到`/etc/gitlab/gitlab.rb` ： 1. 在`gitaly1.internal` ： ``` git_data_dirs ({ 'default' => { 'path' => '/var/opt/gitlab/git-data' }, 'storage1' => { 'path' => '/mnt/gitlab/git-data' }, }) ``` 2. 在`gitaly2.internal` ： ``` git_data_dirs ({ 'storage2' => { 'path' => '/mnt/gitlab/git-data' }, }) ``` 4. 保存文件并[重新配置 GitLab](../restart_gitlab.html#omnibus-gitlab-reconfigure) . 5. 確認 Gitaly 可以執行對內部 API 的回調： ``` sudo /opt/gitlab/embedded/service/gitlab-shell/bin/check -config /opt/gitlab/embedded/service/gitlab-shell/config.yml ``` 6. 驗證 GitLab 服務正在運行： ``` sudo gitlab-ctl status ``` 輸出應類似于以下內容： ``` run: consul: (pid 30339) 77006s; run: log: (pid 29878) 77020s run: gitaly: (pid 30351) 77005s; run: log: (pid 29660) 77040s run: logrotate: (pid 7760) 3213s; run: log: (pid 29782) 77032s run: node-exporter: (pid 30378) 77004s; run: log: (pid 29812) 77026s ``` ### Gitaly TLS support[](#gitaly-tls-support "Permalink") Gitaly 支持 TLS 加密. 為了能夠與偵聽安全連接的 Gitaly 實例進行通信，您將需要在 GitLab 配置中相應存儲條目的`gitaly_address`中使用`tls://` URL 方案. 您將需要攜帶自己的證書，因為該證書不會自動提供. 證書或其證書頒發機構必須按照[GitLab 自定義證書配置中](https://docs.gitlab.com/omnibus/settings/ssl.html)所述的步驟，安裝在所有 Gitaly 節點（包括使用證書的 Gitaly 節點）上，以及與之通信的所有客戶端節點上. **注意：**自簽名證書必須指定用于訪問 Gitaly 服務器的地址. 如果要通過主機名尋址 Gitaly 服務器，則可以為此使用"公用名"字段，也可以將其添加為"使用者備用名". 如果要通過 Gitaly 服務器的 IP 地址對其進行尋址，則必須將其作為主題備用名稱添加到證書中. [gRPC 不支持在證書中使用 IP 地址作為公用名](https://github.com/grpc/grpc/issues/2691) .**注意：**可以同時為 Gitaly 服務器配置未加密的偵聽地址`listen_addr`和已加密的偵聽地址`tls_listen_addr` . 如果需要，這使您可以從未加密的流量逐漸過渡到加密的流量. 要使用 TLS 配置 Gitaly： 1. 創建`/etc/gitlab/ssl`目錄，并在其中復制密鑰和證書： ``` sudo mkdir -p /etc/gitlab/ssl sudo chmod 755 /etc/gitlab/ssl sudo cp key.pem cert.pem /etc/gitlab/ssl/ sudo chmod 644 key.pem cert.pem ``` 2. 將證書復制到`/etc/gitlab/trusted-certs`以便 Gitaly 在調用自身時信任該證書： ``` sudo cp /etc/gitlab/ssl/cert.pem /etc/gitlab/trusted-certs/ ``` 3. 編輯`/etc/gitlab/gitlab.rb`并添加： ``` gitaly['tls_listen_addr'] = "0.0.0.0:9999" gitaly['certificate_path'] = "/etc/gitlab/ssl/cert.pem" gitaly['key_path'] = "/etc/gitlab/ssl/key.pem" ``` 4. 刪除`gitaly['listen_addr']`以僅允許加密連接. 5. 保存文件并[重新配置 GitLab](../restart_gitlab.html#omnibus-gitlab-reconfigure) . [Back to setup components](#setup-components) ## Configure Sidekiq[](#configure-sidekiq "Permalink") Sidekiq 需要連接到 Redis，PostgreSQL 和 Gitaly 實例. 以下 IP 將作為示例： * `10.6.0.71` ：Sidekiq 1 * `10.6.0.72` ：Sidekiq 2 * `10.6.0.73` ：Sidekiq 3 * `10.6.0.74` ：Sidekiq 4 要配置 Sidekiq 節點，每個節點一個： 1. SSH 到 Sidekiq 服務器. 2. 從 GitLab 下載頁面使用步驟 1 和 2 [下載/安裝](https://about.gitlab.com/install/)所需的 Omnibus GitLab 軟件包. **不要完成下載頁面上的任何其他步驟.** 3. 使用編輯器打開`/etc/gitlab/gitlab.rb` ： ``` ######################################## ##### Services Disabled ### ######################################## nginx['enable'] = false grafana['enable'] = false prometheus['enable'] = false gitlab_rails['auto_migrate'] = false alertmanager['enable'] = false gitaly['enable'] = false gitlab_workhorse['enable'] = false nginx['enable'] = false puma['enable'] = false postgres_exporter['enable'] = false postgresql['enable'] = false redis['enable'] = false redis_exporter['enable'] = false gitlab_exporter['enable'] = false ######################################## #### Redis ### ######################################## ## Must be the same in every sentinel node redis['master_name'] = 'gitlab-redis' ## The same password for Redis authentication you set up for the master node. redis['master_password'] = '<redis_primary_password>' ## A list of sentinels with `host` and `port` gitlab_rails['redis_sentinels'] = [ {'host' => '10.6.0.11', 'port' => 26379}, {'host' => '10.6.0.12', 'port' => 26379}, {'host' => '10.6.0.13', 'port' => 26379}, ] ####################################### ### Gitaly ### ####################################### git_data_dirs({ 'default' => { 'gitaly_address' => 'tcp://gitaly1.internal:8075' }, 'storage1' => { 'gitaly_address' => 'tcp://gitaly1.internal:8075' }, 'storage2' => { 'gitaly_address' => 'tcp://gitaly2.internal:8075' }, }) gitlab_rails['gitaly_token'] = 'YOUR_TOKEN' ####################################### ### Postgres ### ####################################### gitlab_rails['db_host'] = '10.6.0.20' # internal load balancer IP gitlab_rails['db_port'] = 6432 gitlab_rails['db_password'] = '<postgresql_user_password>' gitlab_rails['db_adapter'] = 'postgresql' gitlab_rails['db_encoding'] = 'unicode' gitlab_rails['auto_migrate'] = false ####################################### ### Sidekiq configuration ### ####################################### sidekiq['listen_address'] = "0.0.0.0" ####################################### ### Monitoring configuration ### ####################################### consul['enable'] = true consul['monitoring_service_discovery'] = true consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13) } # Set the network addresses that the exporters will listen on node_exporter['listen_address'] = '0.0.0.0:9100' # Rails Status for prometheus gitlab_rails['monitoring_whitelist'] = ['10.6.0.81/32', '127.0.0.0/8'] gitlab_rails['prometheus_address'] = '10.6.0.81:9090' ``` 4. 保存文件并[重新配置 GitLab](../restart_gitlab.html#omnibus-gitlab-reconfigure) . 5. 驗證 GitLab 服務正在運行： ``` sudo gitlab-ctl status ``` 輸出應類似于以下內容： ``` run: consul: (pid 30114) 77353s; run: log: (pid 29756) 77367s run: logrotate: (pid 9898) 3561s; run: log: (pid 29653) 77380s run: node-exporter: (pid 30134) 77353s; run: log: (pid 29706) 77372s run: sidekiq: (pid 30142) 77351s; run: log: (pid 29638) 77386s ``` **提示：**您還可以運行[多個 Sidekiq 進程](../operations/extra_sidekiq_processes.html) .[Back to setup components](#setup-components) ## Configure GitLab Rails[](#configure-gitlab-rails "Permalink") **注意：**在我們的體系結構中，我們使用 Puma Web 服務器運行每個 GitLab Rails 節點，并將其工作程序數設置為可用 CPU 的 90％以及四個線程. 對于運行帶有其他組件的 Rails 的節點，應該相應地降低 worker 的值，我們發現 50％達到了很好的平衡，但這取決于工作量. 本節介紹如何配置 GitLab 應用程序（Rails）組件. 在每個節點上執行以下操作： 1. 如果您[使用的是 NFS](#configure-nfs-optional) ： 1. 如有必要，請使用以下命令安裝 NFS 客戶端實用程序軟件包： ``` # Ubuntu/Debian apt-get install nfs-common # CentOS/Red Hat yum install nfs-utils nfs-utils-lib ``` 2. 在`/etc/fstab`指定必要的 NFS 掛載. `/etc/fstab`的確切內容取決于您選擇配置 NFS 服務器的方式. 有關示例和各種選項，請參見[NFS 文檔](../high_availability/nfs.html) . 3. 創建共享目錄. 這些可能會有所不同，具體取決于您的 NFS 安裝位置. ``` mkdir -p /var/opt/gitlab/.ssh /var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/shared /var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/git-data ``` 2. 使用[GitLab 下載中的](https://about.gitlab.com/install/) **步驟 1 和 2**下載/安裝 Omnibus GitLab. 不要完成下載頁面上的其他步驟. 3. 創建/編輯`/etc/gitlab/gitlab.rb`并使用以下配置. 為了保持整個節點的鏈接均勻性， `external_url`在應用服務器上應指向外部 URL，用戶將用來訪問 GitLab. 這將是[外部負載平衡器](#configure-the-external-load-balancer)的 URL，它將[負載](#configure-the-external-load-balancer)流量路由到 GitLab 應用程序服務器： ``` external_url 'https://gitlab.example.com' # Gitaly and GitLab use two shared secrets for authentication, one to authenticate gRPC requests # to Gitaly, and a second for authentication callbacks from GitLab-Shell to the GitLab internal API. # The following two values must be the same as their respective values # of the Gitaly setup gitlab_rails['gitaly_token'] = 'gitalyecret' gitlab_shell['secret_token'] = 'shellsecret' git_data_dirs({ 'default' => { 'gitaly_address' => 'tcp://gitaly1.internal:8075' }, 'storage1' => { 'gitaly_address' => 'tcp://gitaly1.internal:8075' }, 'storage2' => { 'gitaly_address' => 'tcp://gitaly2.internal:8075' }, }) ## Disable components that will not be on the GitLab application server roles ['application_role'] gitaly['enable'] = false nginx['enable'] = true sidekiq['enable'] = false ## PostgreSQL connection details # Disable PostgreSQL on the application node postgresql['enable'] = false gitlab_rails['db_host'] = '10.6.0.20' # internal load balancer IP gitlab_rails['db_port'] = 6432 gitlab_rails['db_password'] = '<postgresql_user_password>' gitlab_rails['auto_migrate'] = false ## Redis connection details ## Must be the same in every sentinel node redis['master_name'] = 'gitlab-redis' ## The same password for Redis authentication you set up for the Redis primary node. redis['master_password'] = '<redis_primary_password>' ## A list of sentinels with `host` and `port` gitlab_rails['redis_sentinels'] = [ {'host' => '10.6.0.11', 'port' => 26379}, {'host' => '10.6.0.12', 'port' => 26379}, {'host' => '10.6.0.13', 'port' => 26379} ] ## Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters used for monitoring will listen on node_exporter['listen_address'] = '0.0.0.0:9100' gitlab_workhorse['prometheus_listen_addr'] = '0.0.0.0:9229' sidekiq['listen_address'] = "0.0.0.0" puma['listen'] = '0.0.0.0' ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } # Add the monitoring node's IP address to the monitoring whitelist and allow it to # scrape the NGINX metrics gitlab_rails['monitoring_whitelist'] = ['10.6.0.81/32', '127.0.0.0/8'] nginx['status']['options']['allow'] = ['10.6.0.81/32', '127.0.0.0/8'] gitlab_rails['prometheus_address'] = '10.6.0.81:9090' ## Uncomment and edit the following options if you have set up NFS ## ## Prevent GitLab from starting if NFS data mounts are not available ## #high_availability['mountpoint'] = '/var/opt/gitlab/git-data' ## ## Ensure UIDs and GIDs match between servers for permissions via NFS ## #user['uid'] = 9000 #user['gid'] = 9000 #web_server['uid'] = 9001 #web_server['gid'] = 9001 #registry['uid'] = 9002 #registry['gid'] = 9002 ``` 4. 如果您正在使用[具有 TLS 支持](#gitaly-tls-support)的`git_data_dirs` ，請確保`git_data_dirs`條目配置了`tls`而不是`tcp` ： ``` git_data_dirs({ 'default' => { 'gitaly_address' => 'tls://gitaly1.internal:9999' }, 'storage1' => { 'gitaly_address' => 'tls://gitaly1.internal:9999' }, 'storage2' => { 'gitaly_address' => 'tls://gitaly2.internal:9999' }, }) ``` 1. 將證書復制到`/etc/gitlab/trusted-certs` ： ``` sudo cp cert.pem /etc/gitlab/trusted-certs/ ``` 5. 保存文件并[重新配置 GitLab](../restart_gitlab.html#omnibus-gitlab-reconfigure) . 6. 運行`sudo gitlab-rake gitlab:gitaly:check`確認節點可以連接到 Gitaly. 7. 拖尾日志以查看請求： ``` sudo gitlab-ctl tail gitaly ``` 8. 驗證 GitLab 服務正在運行： ``` sudo gitlab-ctl status ``` 輸出應類似于以下內容： ``` run: consul: (pid 4890) 8647s; run: log: (pid 29962) 79128s run: gitlab-exporter: (pid 4902) 8647s; run: log: (pid 29913) 79134s run: gitlab-workhorse: (pid 4904) 8646s; run: log: (pid 29713) 79155s run: logrotate: (pid 12425) 1446s; run: log: (pid 29798) 79146s run: nginx: (pid 4925) 8646s; run: log: (pid 29726) 79152s run: node-exporter: (pid 4931) 8645s; run: log: (pid 29855) 79140s run: puma: (pid 4936) 8645s; run: log: (pid 29656) 79161s ``` **注意：**如上例所示，當在`external_url`指定`https`時，GitLab 會假定您在`/etc/gitlab/ssl/`具有 SSL 證書. 如果沒有證書，NGINX 將無法啟動. 有關更多信息，請參見[NGINX 文檔](https://docs.gitlab.com/omnibus/settings/nginx.html) . ### GitLab Rails post-configuration[](#gitlab-rails-post-configuration "Permalink") 1. 確保運行所有遷移： ``` gitlab-rake gitlab:db:configure ``` **注意：**如果遇到`rake aborted!` 錯誤，指出 PgBouncer 是無法連接到 PostgreSQL 也可能是您的 PgBouncer 節點的 IP 地址是從 PostgreSQL 的缺失`trust_auth_cidr_addresses`在`gitlab.rb`你的數據庫節點. 請參閱"故障排除"部分中的[PgBouncer 錯誤`ERROR: pgbouncer cannot connect to server`](troubleshooting.html#pgbouncer-error-error-pgbouncer-cannot-connect-to-server) ，然后再繼續. 2. [Configure fast lookup of authorized SSH keys in the database](../operations/fast_ssh_key_lookup.html). [Back to setup components](#setup-components) ## Configure Prometheus[](#configure-prometheus "Permalink") Omnibus GitLab 軟件包可用于配置運行[Prometheus](../monitoring/prometheus/index.html)和[Grafana](../monitoring/performance/grafana_configuration.html)的獨立 Monitoring 節點： 1. SSH 進入"監視"節點. 2. 從 GitLab 下載頁面使用**步驟 1 和 2** [下載/安裝](https://about.gitlab.com/install/)所需的 Omnibus GitLab 軟件包. 不要完成下載頁面上的任何其他步驟. 3. 編輯`/etc/gitlab/gitlab.rb`并添加內容： ``` external_url 'http://gitlab.example.com' # Disable all other services gitlab_rails['auto_migrate'] = false alertmanager['enable'] = false gitaly['enable'] = false gitlab_exporter['enable'] = false gitlab_workhorse['enable'] = false nginx['enable'] = true postgres_exporter['enable'] = false postgresql['enable'] = false redis['enable'] = false redis_exporter['enable'] = false sidekiq['enable'] = false puma['enable'] = false unicorn['enable'] = false node_exporter['enable'] = false gitlab_exporter['enable'] = false # Enable Prometheus prometheus['enable'] = true prometheus['listen_address'] = '0.0.0.0:9090' prometheus['monitor_kubernetes'] = false # Enable Login form grafana['disable_login_form'] = false # Enable Grafana grafana['enable'] = true grafana['admin_password'] = '<grafana_password>' # Enable service discovery for Prometheus consul['enable'] = true consul['monitoring_service_discovery'] = true consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13) } ``` 4. 保存文件并[重新配置 GitLab](../restart_gitlab.html#omnibus-gitlab-reconfigure) . 5. 在 GitLab 用戶界面中，將`admin/application_settings/metrics_and_profiling` >指標-Grafana 設置為`/-/grafana`到`http[s]://<MONITOR NODE>/-/grafana` . 6. 驗證 GitLab 服務正在運行： ``` sudo gitlab-ctl status ``` 輸出應類似于以下內容： ``` run: consul: (pid 31637) 17337s; run: log: (pid 29748) 78432s run: grafana: (pid 31644) 17337s; run: log: (pid 29719) 78438s run: logrotate: (pid 31809) 2936s; run: log: (pid 29581) 78462s run: nginx: (pid 31665) 17335s; run: log: (pid 29556) 78468s run: prometheus: (pid 31672) 17335s; run: log: (pid 29633) 78456s ``` [Back to setup components](#setup-components) ## Configure the object storage[](#configure-the-object-storage "Permalink") GitLab 支持使用對象存儲服務來保存多種類型的數據. 建議在[NFS](#configure-nfs-optional)上使用它，并且通常在較大的設置中更好，因為對象存儲通常具有更高的性能，可靠性和可伸縮性. manbetx 客戶端打不開已經測試過或知道使用的客戶的對象存儲選項包括： * SaaS / Cloud 解決方案，例如[Amazon S3](https://aws.amazon.com/s3/) ， [Google 云存儲](https://cloud.google.com/storage) . * 來自各種存儲供應商的本地硬件和設備. * MinIO. 我們的 Helm Chart 文檔中[提供了有關部署的指南](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) . 要配置 GitLab 以使用對象存儲，請根據要使用的功能參考以下指南： 1. Configure [object storage for backups](../../raketasks/backup_restore.html#uploading-backups-to-a-remote-cloud-storage). 2. Configure [object storage for job artifacts](../job_artifacts.html#using-object-storage) including [incremental logging](../job_logs.html#new-incremental-logging-architecture). 3. Configure [object storage for LFS objects](../lfs/index.html#storing-lfs-objects-in-remote-object-storage). 4. Configure [object storage for uploads](../uploads.html#using-object-storage-core-only). 5. Configure [object storage for merge request diffs](../merge_request_diffs.html#using-object-storage). 6. 配置[容器注冊表的對象存儲](../packages/container_registry.html#use-object-storage) （可選功能）. 7. [為 Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)配置[對象存儲](https://docs.mattermost.com/administration/config-settings.html#file-storage) （可選功能）. 8. 配置[軟件包的對象存儲](../packages/index.html#using-object-storage) （可選功能）. 9. 配置[依賴項代理的對象存儲](../packages/dependency_proxy.html#using-object-storage) （可選功能）. 10. [為 Pseudonymizer](../pseudonymizer.html#configuration) （可選功能）配置[對象存儲](../pseudonymizer.html#configuration) . 11. 配置[對象存儲以自動縮放 Runner 緩存](https://docs.gitlab.com/runner/configuration/autoscale.html) （可選-為了提高性能）. 12. Configure [object storage for Terraform state files](../terraform_state.html#using-object-storage-core-only). 對于 GitLab，建議為每種數據類型使用單獨的存儲桶. 我們的配置的局限性是對象存儲的每次使用都是單獨配置的. [我們有一個需要改進的問題](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) ，輕松地將一個存儲桶與單獨的文件夾一起使用可能會帶來一個改進. 使用同一個存儲桶至少有一個特定的問題：當使用 Helm 圖表部署 GitLab 時，除非使用單獨的存儲桶，否則從備份還原[將無法正常工作](https://docs.gitlab.com/charts/advanced/external-object-storage/) . 如果您的組織將來決定將 GitLab 遷移到 Helm 部署，則使用單個存儲桶的一種風險是. GitLab 可以運行，但是直到組織對備份起作用的關鍵要求之前，備份的情況可能無法實現. [Back to setup components](#setup-components) ## Configure NFS (optional)[](#configure-nfs-optional "Permalink") 建議盡可能在 NFS 上使用[對象存儲](#configure-the-object-storage)以及[Gitaly](#configure-gitaly) ，以提高性能. 如果您打算使用 GitLab 頁面，則當前[需要 NFS](troubleshooting.html#gitlab-pages-requires-nfs) . 請參閱如何[配置 NFS](../high_availability/nfs.html) . [Back to setup components](#setup-components) ## Troubleshooting[](#troubleshooting "Permalink") 請參閱[故障排除文檔](troubleshooting.html) . [Back to setup components](#setup-components)