MCollective+MQ架構的部署 · Puppet運維實戰

#### MCollective架構篇2-MCollective+MQ架構的部署 # 1 Mcollective介紹 MCollective 是一個構建服務器編排(Server Orchestration)和并行工作執行系統的框架。首先，MCollective 是一種針對服務器集群進行可編程控制的系統管理解決方案。在這一點上，它的功能類似：Func，Fabric 和 Capistrano。其次，MCollective 的設計打破基于中心存儲式系統和像 SSH 這樣的工具，不再僅僅癡迷于 SSH 的 For 循環。它使用發布訂閱中間件(Publish Subscribe Middleware)這樣的現代化工具和通過目標數據(meta data)而不是主機名(hostnames)來實時發現網絡資源這樣的現代化理念。提供了一個可擴展的而且迅速的并行執行環境。 MCollective 工具為命令行界面，但它可與數千個應用實例進行通信，而且傳輸速度驚人。無論部署的實例位于什么位置，通信都能以線速進行傳輸，使用的是一個類似多路傳送的推送信息系統。MCollective 工具沒有可視化用戶界面，用戶只能通過檢索來獲取需要應用的實例。Puppet Dashboard 提供有這部分功能。 # 2 安裝和配置RabbitMQ ### 2.1 安裝和配置RabbitMQ **2.1.1 安裝RabbitMQ** ~~~ [root@puppetserver rpms]# yum install erlang #RabbitMQ依賴erlang語言，需要安裝大概65個左右的erlang依賴包 [root@puppetserver rpms]# yum install rabbitmq-server [root@puppetserver rpms]# ll /usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/plugins/ #默認已經安裝了stomp插件，老版本需要下載安裝 -rw-r--r-- 1 root root 242999 Aug 24 17:42 amqp_client-3.1.5.ez -rw-r--r-- 1 root root 85847 Aug 24 17:42 rabbitmq_stomp-3.1.5.ez … ~~~ **2.1.2 啟動rabbitmq-server** ~~~ [root@puppetserver rpms]# /etc/rc.d/init.d/rabbitmq-server start #啟動rabbitmq服務 Starting rabbitmq-server: SUCCESS rabbitmq-server. [root@puppetserver rpms]# /etc/rc.d/init.d/rabbitmq-server status #查看rabbitmq狀態 Status of node rabbit@puppetserver ... [{pid,43198}, {running_applications,[{rabbit,"RabbitMQ","3.1.5"}, {mnesia,"MNESIA CXC 138 12","4.5"}, {os_mon,"CPO CXC 138 46","2.2.7"}, {xmerl,"XML parser","1.2.10"}, {sasl,"SASL CXC 138 11","2.1.10"}, {stdlib,"ERTS CXC 138 10","1.17.5"}, {kernel,"ERTS CXC 138 10","2.14.5"}]}, {os,{unix,linux}}, {erlang_version,"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [rq:1] [async-threads:30] [kernel-poll:true]\n"}, {memory,[{total,27101856}, {connection_procs,2648}, {queue_procs,5296}, {plugins,0}, {other_proc,9182320}, {mnesia,57456}, {mgmt_db,0}, {msg_index,21848}, {other_ets,765504}, {binary,3296}, {code,14419185}, {atom,1354457}, {other_system,1289846}]}, {vm_memory_high_watermark,0.4}, {vm_memory_limit,838362726}, {disk_free_limit,1000000000}, {disk_free,15992676352}, {file_descriptors,[{total_limit,924}, {total_used,3}, {sockets_limit,829}, {sockets_used,1}]}, {processes,[{limit,1048576},{used,122}]}, {run_queue,0}, {uptime,4}] ...done. [root@puppetserver rpms]# netstat -nlp | grep beam #默認監聽端口為5672 tcp 0 0 0.0.0.0:44422 0.0.0.0:* LISTEN 43198/beam tcp 0 0 :::5672 :::* LISTEN 43198/beam ~~~ **2.1.3 配置RabbitMQ** **2.1.3.1 加載amqp_client和rabbit_stomp插件** ~~~ [root@puppetserver sbin]#ln -s /usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/sbin/rabbitmq-plugins /usr/sbin/rabbitmq-plugins #創建命令rabbitmq-plugins的軟連接 [root@puppetserver sbin]# ln -s /usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/sbin/rabbitmq-env /usr/sbin/rabbitmq-env #創建命令rabbitmq-env的軟連接 [root@puppetserver sbin]# rabbitmq-plugins enable rabbitmq_stomp #開啟rabbitmq_stomp插件 The following plugins have been enabled: amqp_client rabbitmq_stomp Plugin configuration has changed. Restart RabbitMQ for changes to take effect. [root@puppetserver sbin]# /etc/rc.d/init.d/rabbitmq-server restart Restarting rabbitmq-server: SUCCESS rabbitmq-server. [root@puppetserver rabbitmq]# tailf /var/log/rabbitmq/rabbit\@puppetserver.log #可以從日志看到stomp插件加載成功 =INFO REPORT==== 3-Oct-2013::20:25:18 === started STOMP TCP Listener on [::]:61613 =INFO REPORT==== 3-Oct-2013::20:25:18 === Server startup complete; 2 plugins started. * amqp_client * rabbitmq_stomp ~~~ - **2.1.3.2 創建rabbitmq.config配置文件，修改監聽端口為61613** [root@puppetserver rpms]# vim /etc/rabbitmq/rabbitmq.config [ ~~~ {stomp,[ {tcp_listeners, [61613]} ]} #設置connector為stomp，監聽端口為61613 ~~~ ]. [root@puppetserver rpms]# /etc/rc.d/init.d/rabbitmq-server restart Restarting rabbitmq-server: SUCCESS rabbitmq-server. [root@puppetserver rpms]# netstat -nlp | grep beam #默認監聽端口為61613 tcp 0 0 0.0.0.0:56532 0.0.0.0:* LISTEN 1906/beam.smp tcp 0 0 :::61613 :::* LISTEN 1906/beam.smp tcp 0 0 :::5672 :::* LISTEN 1906/beam.smp **2.1.3.3 刪除默認賬戶guest，為MCollective創建賬戶“mcollective”并設置密碼為“secret”，然后設置權限。** ~~~ [root@puppetserver rpms]# rabbitmqctl delete_user guest Deleting user "guest" ... ...done. [root@puppetserver rpms]# rabbitmqctl add_user mcollective secret Creating user "mcollective" ... ...done. [root@puppetserver rpms]# rabbitmqctl set_permissions -p "/" mcollective ".*" ".*" ".*" Setting permissions for user "mcollective" in vhost "/" ... ...done. [root@puppetserver sbin]# rabbitmqctl list_users #查看監聽用戶 Listing users ... mcollective [] ...done. ~~~ **備注：**RabbitMQ擁有一個默認的guest賬戶，它默認對消息隊列擁有全部權限。出于安全方面的考慮，建議刪除這個賬戶。更多詳細配置信息請參考 [http://www.rabbitmq.com/admin-guide.html](http://www.rabbitmq.com/admin-guide.html) 更多詳細配置信息請參考： [http://docs.puppetlabs.com/mcollective/reference/plugins/connector_rabbitmq.html](http://docs.puppetlabs.com/mcollective/reference/plugins/connector_rabbitmq.html) # 3 安裝和配置MCollective ### 3.1 安裝MCollective **3.1.1 測試端安裝MCollective客戶端** ~~~ [root@puppetserver rpms]# yum install mcollective-common mcollective-client #依賴包rubygem-stomp ~~~ **3.1.2 節點安裝MCollective服務端** ~~~ [root@agent1 ~]# yum install mcollective mcollective-common #依賴rubygem-stomp、rubygems和ruby相關包 ~~~ ### 3.2 配置MCollective **3.2.1 測試端配置MCollective客戶端** ~~~ [root@puppetserver rpms]# vim /etc/mcollective/client.cfg topicprefix = /topic/ main_collective = mcollective collectives = mcollective libdir = /usr/libexec/mcollective logger_type = console loglevel = warn # Plugins securityprovider = psk plugin.psk = a36cd839414370e10fd281b8a38a4f48 #MCollective通信共享密鑰，和MCollective服務端保持一致 connector = stomp #通信協議 plugin.stomp.host = 192.168.100.110 #Middleware地址 plugin.stomp.port = 61613 #Middleware監聽端口 plugin.stomp.user = mcollective #Middleware通信賬號 plugin.stomp.password = secret #Middleware通信密碼 # Facts factsource = yaml plugin.yaml = /etc/mcollective/facts.yaml ~~~ **3.2.2 節點配置MCollective服務端** ~~~ [root@agent1 rpms]# vim /etc/mcollective/server.cfg topicprefix = /topic/ main_collective = mcollective collectives = mcollective libdir = /usr/libexec/mcollective #存放plugins的位置 logfile = /var/log/mcollective.log loglevel = info daemonize = 1 # Plugins securityprovider = psk plugin.psk = a36cd839414370e10fd281b8a38a4f48 #MCollective通信共享密鑰，和MCollective客戶端保持一致 connector = stomp #通信協議 plugin.stomp.host = 192.168.100.110 #Middleware地址 plugin.stomp.port = 61613 #Middleware監聽端口 plugin.stomp.user = mcollective #Middleware通信賬號 plugin.stomp.password = secret #Middleware通信密碼 # Facts factsource = yaml plugin.yaml = /etc/mcollective/facts.yaml [root@agent1 ~]# /etc/rc.d/init.d/mcollective start Starting mcollective: [ OK ] [root@agent1 ~]# chkconfig mcollective on [root@agent1 ~]# ~~~ ### 3.3 測試Mcollective與Middleware通信 ~~~ [root@puppetserver rpms]# mco ping #檢查所有存活的節點 agent2.kisspuppet.com time=119.98 ms agent1.kisspuppet.com time=159.31 ms ---- ping statistics ---- 2 replies max: 159.31 min: 119.98 avg: 139.64 [root@puppetserver rpms]# mco find agent1.kisspuppet.com agent2.kisspuppet.com ~~~