## 集群是分布式架構 1. 集群情況下每個節點之間會互相通信的; 2. 集群情況下每個節點都是可以讀寫的; 3. cluster 中Redis只有DB0沒有DB1-15  ## Redis cluster架構 1. 節點 2. meet 3. 指派槽 4. 復制(每個主節點都有一個從節點) ## 節點 ``` cluster-enabled:yes //集群模式啟動 ``` ## meet 每個節點和其他節點都是互相通信的;所有節點共享信息,這樣每個節點才能知道哪個節點負責哪些槽位;  ## 指派槽 ## Redis cluster特性 1. 復制(每個主節點都有從節點) 2. 高可用 3. 分片 # Redis cluster安裝配置 ## 原生命令安裝-理解架構 1. 配置開啟節點,最后一項配置是不合理的,正式項目要設置為no;  主要配置  2. meet ``` cluster meet ip port ``` 3. 指派槽 ``` cluster addslots slot [slot...] ``` 4. 主從關系 ``` cluster replicate node-id ``` ## 完整配置 將端口進行替換 ``` port 7001 dbfilename dump-7001.rdb cluster-enabled yes cluster-config-file nodes-7001.conf cluster-node-timeout 15000 ``` 查看進程 ``` ? ~ ps -ef | grep redis 501 9243 1376 0 10:47下午 ttys000 0:00.22 ./redis-server 127.0.0.1:7000 [cluster] 501 9356 2784 0 10:51下午 ttys001 0:00.04 ./redis-server 127.0.0.1:7001 [cluster] 501 9358 3064 0 10:51下午 ttys002 0:00.03 ./redis-server 127.0.0.1:7002 [cluster] 501 9360 3120 0 10:51下午 ttys003 0:00.02 ./redis-server 127.0.0.1:7003 [cluster] 501 9365 4897 0 10:51下午 ttys005 0:00.02 ./redis-server 127.0.0.1:7004 [cluster] 501 9368 5230 0 10:51下午 ttys006 0:00.02 ./redis-server 127.0.0.1:7005 [cluster] ``` 連接節點,輸入命令發現不可用 ``` ? bin ./redis-cli -p 7000 127.0.0.1:7000> set name jack (error) CLUSTERDOWN Hash slot not served //不可用 ``` ``` ? bin ./redis-cli -p 7000 cluster nodes 8081564a18dc03a9d4384ed502fa40de22ca57fb :7000@17000 myself,master - 0 0 0 connected ``` 集群信息 ``` ? bin ./redis-cli -p 7000 cluster info cluster_state:fail //狀態是失敗的 cluster_slots_assigned:0 //分配的槽 cluster_slots_ok:0 //成功的槽 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:1 cluster_size:0 cluster_current_epoch:0 cluster_my_epoch:0 cluster_stats_messages_sent:0 cluster_stats_messages_received:0 ``` 集群配置文件,也就是cluster-config-file 配置的文件: ``` ? redis cat nodes-7000.conf 8081564a18dc03a9d4384ed502fa40de22ca57fb :0@0 myself,master - 0 0 0 connected vars currentEpoch 0 lastVoteEpoch 0 ``` * **集群模式的Redis除了原有的配置文件之外又加了一份集群配置文件**。 當集群內節點信息發生變化，如添加節點、節點下線、故障轉移等。節點會自動保存集群狀態到配置文件中。需要注意的是，**Redis自動維護集群配置文件，不要手動修改，**防止節點重啟時產生集群信息錯亂 * 6379節點啟動成功，**第一次啟動時如果沒有集群配置文件，它會自動創建一份，**文件名稱采用cluster-config-file參數項控制，建議采用node- {port}.conf格式定義，通過使用端口號區分不同節點，防止同一機器下多個 節點彼此覆蓋，造成集群信息異常 * **如果啟動時存在集群配置文件，節點會使用配置文件內容初始化集群信息**。啟動過程如下圖所示：  * 文件內容記錄了集群初始狀態，**這里最重要的是節點ID，它是一個40位16進制字符串，用于唯一標識集群內一個節點**，之后很多集群操作都要借助于節點ID來完成。**需要注意是，節點ID不同于運行ID：**節點ID在集群初始化 時只創建一次，節點重啟時會加載集群配置文件進行重用，而Redis的運行 ID每次重啟都會變化 * **在節點6380執行cluster nodes命令獲取集群節點狀態，**每個節點目前只能識別出自己的節點信息; ## meet 操作(握手) * 我們啟動6個節點，但每個節點彼此并不知道對方的存在，**下面通過節點握手讓6個節點彼此建立聯系從而組成一個集群**; ``` ? bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7001 //meet操作 OK ``` 執行cluster nodes可以看到握手成功 ``` ? bin ./redis-cli -p 7000 cluster nodes a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606057637751 0 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 myself,master - 0 0 1 connected ``` ``` ? bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7002 OK ? bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7003 OK ? bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7004 OK ? bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7005 OK ? bin ./redis-cli -p 7003 cluster nodes //可以看到該節點發現了其他節點,我們只需要在集群內任意節點上執行cluster meet命令加入新節點，握手狀態會通過消息在集群內傳播,這樣其他節點會自動發現新節點并發起握手流程 f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 myself,master - 0 1606057695000 0 connected e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 master - 0 1606057695000 5 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 master - 0 1606057695000 4 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606057696128 1 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 master - 0 1606057695104 3 connected a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606057697157 2 connected ``` ## 指派插槽 集群彼此發現,但是沒有分配插槽; ``` ? bin ./redis-cli -p 7003 cluster nodes f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 myself,master - 0 1606057695000 0 connected e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 master - 0 1606057695000 5 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 master - 0 1606057695000 4 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606057696128 1 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 master - 0 1606057695104 3 connected a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606057697157 2 connected ``` 指派插槽: ``` ? bin ./redis-cli -p 7000 cluster addslots {0..5461} OK ? bin ./redis-cli -p 7001 cluster addslots {5462..10922} OK ? bin ./redis-cli -p 7002 cluster addslots {10923..16383} OK ``` ``` ? bin ./redis-cli -p 7002 cluster nodes e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 slave a4890ef360af0f5d9e772f5523ec46aed93af198 0 1606059393911 5 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606059393000 1 connected 0-5461 //0-5461就是分配的插槽 a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606059390875 2 connected 5462-10922 f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 slave 8081564a18dc03a9d4384ed502fa40de22ca57fb 0 1606059391000 1 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 slave e1d86f41ed70cf14250d6727643a4828bc7acaf1 0 1606059392894 4 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 myself,master - 0 1606059391000 3 connected 10923-16383 ``` 查看插槽: ``` ? bin ./redis-cli -p 7002 cluster slots 1) 1) (integer) 0 2) (integer) 5461 3) 1) "127.0.0.1" 2) (integer) 7000 3) "8081564a18dc03a9d4384ed502fa40de22ca57fb" 4) 1) "127.0.0.1" 2) (integer) 7003 3) "f4956ec33caae7ab7ac00fa7bfac56d338f6cd87" 2) 1) (integer) 5462 2) (integer) 10922 3) 1) "127.0.0.1" 2) (integer) 7001 3) "a4890ef360af0f5d9e772f5523ec46aed93af198" 4) 1) "127.0.0.1" 2) (integer) 7004 3) "e8a77578af06d00e95cffd7059313c979f208212" 3) 1) (integer) 10923 2) (integer) 16383 3) 1) "127.0.0.1" 2) (integer) 7002 3) "e1d86f41ed70cf14250d6727643a4828bc7acaf1" 4) 1) "127.0.0.1" 2) (integer) 7005 3) "a056926a23dc3a62c8da560c9ac6b4e29fd657cf" ``` ## 主從分配 我們的目的是7003對應7000,7004對應7001,7005對應7002; ``` ? bin ./redis-cli -p 7003 cluster replicate 8081564a18dc03a9d4384ed502fa40de22ca57fb //cluster replicate 主節點的node id OK ? bin ./redis-cli -p 7004 cluster replicate a4890ef360af0f5d9e772f5523ec46aed93af198 OK ? bin ./redis-cli -p 7005 cluster replicate e1d86f41ed70cf14250d6727643a4828bc7acaf1 OK ``` 查看主從關系: ``` ? bin ./redis-cli -p 7003 cluster nodes f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 myself,slave 8081564a18dc03a9d4384ed502fa40de22ca57fb 0 1606058562000 0 connected //可以看到是8081564a18dc03a9d4384ed502fa40de22ca57fb的從節點 e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 slave a4890ef360af0f5d9e772f5523ec46aed93af198 0 1606058562031 5 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 slave e1d86f41ed70cf14250d6727643a4828bc7acaf1 0 1606058562000 4 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606058561017 1 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 master - 0 1606058563046 3 connected a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606058564072 2 connected ``` ## 使用集群 ``` ? bin ./redis-cli -c -p 7000 127.0.0.1:7000> set name jack -> Redirected to slot [5798] located at 127.0.0.1:7001 OK 127.0.0.1:7001> get name "jack" ``` ## 官方工具安裝 1. 安裝ruby; 2. 安裝rubygem redis; 3. 安裝redis-trib.rb; ## 使用redis-trib進行安裝 * redis-cli --cluster命令本來是由redis-trib.rb工具提供的，但是隨著發展，redis-trib.rb工具的功能被逐漸歸納到redis-cli工具中了,但是在Redis5.0版本以上才支持; 1. 所有的Redis.conf配合和手動的配置是一致的; 2. 將所有的Redis-server 啟動; 3. 注意,節點下線需要刪除配置或者啟用新的端口(這里指的是cluster-config-file的配置文件). **輸入下面的命令自動完成節點握手和槽分配。命令如下：** * \--cluster-replicas 1：指定集群中每個主節點配備幾個從節點，這里設置為1。并且該命令會自己創建主節點和分配從節點，其中前3個是主節點，后3個是從節點，后3個從節點分別復制前3個主節點 ``` ./redis-cli --cluster create --cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 ``` * 輸入上面的命令之后，中間會讓你輸入yes或no，下面我們輸入yes開始執行節點握手和槽分配操作 ``` ? bin ./redis-cli --cluster create --cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 127.0.0.1:7004 to 127.0.0.1:7000 Adding replica 127.0.0.1:7005 to 127.0.0.1:7001 Adding replica 127.0.0.1:7003 to 127.0.0.1:7002 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: e4bba455615996a2419f5ea68fb483572da69a4e 127.0.0.1:7000 slots:[0-5460] (5461 slots) master M: 26cdec411e03740e81741fd1af56c84a8995e1bf 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master M: 86ebe5a560dc152174ae87b4555890486aa35051 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master S: 6f0a7298bd81de901ebd4989504a4d914e5002e2 127.0.0.1:7003 replicates 86ebe5a560dc152174ae87b4555890486aa35051 S: f29d5a33719410d0d965f0cd04a1aaf62772113c 127.0.0.1:7004 replicates e4bba455615996a2419f5ea68fb483572da69a4e S: 4c0dcdf4862435f9e0b7ab9573c5d88ebce69c9e 127.0.0.1:7005 replicates 26cdec411e03740e81741fd1af56c84a8995e1bf Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join . >>> Performing Cluster Check (using node 127.0.0.1:7000) M: e4bba455615996a2419f5ea68fb483572da69a4e 127.0.0.1:7000 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: 26cdec411e03740e81741fd1af56c84a8995e1bf 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 6f0a7298bd81de901ebd4989504a4d914e5002e2 127.0.0.1:7003 slots: (0 slots) slave replicates 86ebe5a560dc152174ae87b4555890486aa35051 S: 4c0dcdf4862435f9e0b7ab9573c5d88ebce69c9e 127.0.0.1:7005 slots: (0 slots) slave replicates 26cdec411e03740e81741fd1af56c84a8995e1bf M: 86ebe5a560dc152174ae87b4555890486aa35051 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: f29d5a33719410d0d965f0cd04a1aaf62772113c 127.0.0.1:7004 slots: (0 slots) slave replicates e4bba455615996a2419f5ea68fb483572da69a4e [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. ```  * 最后的輸出報告說明：16384個槽全部被分配，集群創建成功。這里需要注意命令中節點的地址必須是不包含任何槽/數據的節點，否則會拒絕創建集群 * **備注：**如果只想創建主節點，而不同時創建從節點，那么需要忽略--cluster-replicas 1參數。命令如下，例如下面只創建3個集群主節點; ``` redis-cli --cluster create 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:638 ``` ## 集群完整性檢查 * 集群完整性指所有的槽都分配到存活的主節點上，只要16384個槽中有 一個沒有分配給節點則表示集群不完整 * 可以使用下面的命令檢測之前創建的兩個集群是否成功，check命令只需要給出集群中任意一個節點地址就可以完成整個集群的檢查工作： ``` ./redis-cli --cluster check 127.0.0.1:7000 ``` 檢查結果 ``` ? bin ./redis-cli --cluster check 127.0.0.1:7000 127.0.0.1:7000 (e4bba455...) -> 0 keys | 5461 slots | 1 slaves. 127.0.0.1:7001 (26cdec41...) -> 0 keys | 5462 slots | 1 slaves. 127.0.0.1:7002 (86ebe5a5...) -> 0 keys | 5461 slots | 1 slaves. [OK] 0 keys in 3 masters. 0.00 keys per slot on average. >>> Performing Cluster Check (using node 127.0.0.1:7000) M: e4bba455615996a2419f5ea68fb483572da69a4e 127.0.0.1:7000 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: 26cdec411e03740e81741fd1af56c84a8995e1bf 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 6f0a7298bd81de901ebd4989504a4d914e5002e2 127.0.0.1:7003 slots: (0 slots) slave replicates 86ebe5a560dc152174ae87b4555890486aa35051 S: 4c0dcdf4862435f9e0b7ab9573c5d88ebce69c9e 127.0.0.1:7005 slots: (0 slots) slave replicates 26cdec411e03740e81741fd1af56c84a8995e1bf M: 86ebe5a560dc152174ae87b4555890486aa35051 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: f29d5a33719410d0d965f0cd04a1aaf62772113c 127.0.0.1:7004 slots: (0 slots) slave replicates e4bba455615996a2419f5ea68fb483572da69a4e [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. ``` ## 總結 1. 原生命令安裝: 1. 理解Redis cluster; 2. 生產環境不使用(繁瑣,產生錯誤的概率較大); 2. 官方工具安裝 1. 高效,準確; 3. 其他 1.可視化部署