前面我們已經部署了單節點的OpenStack環境，這次我們嘗試部署一個多節點的環境(包括1個controller節點，1個compute節點，1個storage節點)，之前的單節點因為已經安裝了kolla-ansible和docker registry，我們把它當做部署節點。 # 環境準備 準備三臺服務器，分別作為controller節點，compute節點，storage節點。其中controller節點3張網卡，compute、storage節點2張網卡。操作系統為centos7.4 1）關閉Selinux vi /etc/sysconfig/selinux SELINUX=disabled 2）關閉firewalld systemctl stop firewalld systemctl disable firewalld 3）設置主機名，hosts文件 vi /etc/hosts 172.16.31.130 control01 172.16.31.131 controller 172.16.31.132 compute 172.16.31.133 storage 4）storage節點加盤 為storage節點劃分磁盤，作為cinder的lvm后端，storage節點有13tb的硬盤采用parted分區，分區過程略去  創建pv、vg [root@storage ~]# pvcreate /dev/sdb Physical volume "/dev/sdb" successfully created. [root@storage ~]# vgcreate cinder-volumes /dev/sdb Volume group "cinder-volumes" successfully created 修改配置 vi /etc/lvm/lvm.conf  重啟lvm服務 systemctl restart lvm2-lvmetad.service 5）確認節點配置 controller 3 張網卡 compute 2 張網卡 storage 2 張網卡(其實一張即可，因為不需要tunnel網絡) storage 磁盤已經分區  # 安裝docker 1）下載相關docker rpm包 這里我已經到docker把docker的rpm下載下來了，我們上傳的相關的目錄下，當然還是需要配centos和epel的源才能正常安裝的  2）安裝Docker 1.12.6（三臺服務器都要安裝） yum install docker-engine-* -y 在各個節點安裝 yum install python-docker-py -y 3）設置Docker mkdir /etc/systemd/system/docker.service.d tee /etc/systemd/system/docker.service.d/kolla.conf << 'EOF' [Service] MountFlags=shared EOF 4）重啟相關服務 systemctl daemon-reload systemctl enable docker systemctl restart docker 5）編輯/usr/lib/systemd/system/docker.service文件 ExecStart=/usr/bin/dockerd --insecure-registry 172.16.31.130:4000 6）重啟Docker服務 systemctl daemon-reload systemctl restart docker # 配置免密登錄 部署節點和其他節點免密登陸 [root@control01 ~]# ssh-keygen [root@control01 ~]# ssh-copy-id controller [root@control01 ~]# ssh-copy-id compute [root@control01 ~]# ssh-copy-id storage # 部署OpenStack 1）停止部署節點容器 因為部署節點之前已經部署了All in One的OpenStack，為了降低資源占用，我們可以先停止本節的OpenStack相關容器。 [root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/ [root@control01 tools]# . kolla-ansible stop 2）修改multinode文件 [root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/ [root@control01 kolla-ansible-4.0.3.dev36]# cp ansible/inventory/* /home/ [root@control01 ~]# cd /home [root@control01 home]# ls all-in-one multinode [root@control01 home]# cp multinode mymultinode [root@control01 home]# vi mymultinode [control] controller [network] controller [compute] compute [monitoring] controller [storage] storage ...... 里面的內容很好理解，表示一個控制節點 controller，網絡節點也安裝到控制節點，一個計算節點 compute ，一個存儲節點 storage，后面的部分不用修改 3）修改global和password配置文件 [root@control01 ~]# cd /etc/kolla/ [root@control01 kolla]# vi globals.yml ————————————分割線——————————————— kolla_internal_vip_address: "172.16.31.131" 外部訪問地址，如果是非HA環境，是控制節點IP docker_registry: "172.16.31.130:4000" 鏡像庫，用部署節點的鏡像庫 docker_namespace: "99cloud" network_interface: "eno1" 網絡接口，默認就這一個接口，這里我們以eth0作為API網絡 api_interface: "{{ network_interface }}" 用的是eno2 tunnel_interface: "eno2" eno2作為vxlan的承載網絡 neutron_external_interface: "eno3" eno3作為外部網絡 enable_cinder: "yes" 啟用cinder enable_cinder_backend_lvm: "yes" cinder后端用lvm cinder_volume_group: "cinder-volumes" enable_haproxy: "no" 不啟用haproxy ————————————分割線——————————————— [root@control01 kolla]# vi passwords.yml keystone_admin_password: admin admin用戶登錄密碼 4）部署前檢查 [root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/ [root@control01 ~]# . kolla-ansible prechecks -i /home/mymultinode 沒有報錯即可正常部署 5）部署 [root@control01 tools]# . kolla-ansible deploy -i /home/mymultinode 開始部署，等待大約20分鐘，部署完成，一般不會有什么問題，如果有問題，則需要根據報錯及容器日志查找原因  登錄dashboard http://172.16.31.131  登錄系統，查看各個服務正常，表明部署成功。 可以通過docker ps命令查看各個節點運行的容器 controller節點： [root@controller ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1ecab1b4246e 172.16.31.130:4000/99cloud/centos-source-horizon:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours horizon b264ea5467ff 172.16.31.130:4000/99cloud/centos-source-heat-engine:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours heat_engine 08ab0aa8603c 172.16.31.130:4000/99cloud/centos-source-heat-api-cfn:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours heat_api_cfn 6f6aba4804a2 172.16.31.130:4000/99cloud/centos-source-heat-api:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours heat_api 67729f946b23 172.16.31.130:4000/99cloud/centos-source-neutron-metadata-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_metadata_agent f8f8d9f766e9 172.16.31.130:4000/99cloud/centos-source-neutron-l3-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_l3_agent fd3a8ca63fdf 172.16.31.130:4000/99cloud/centos-source-neutron-dhcp-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_dhcp_agent c9f9f72090a4 172.16.31.130:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_openvswitch_agent 9f19bca0759c 172.16.31.130:4000/99cloud/centos-source-neutron-server:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_server e1286083b3f7 172.16.31.130:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_vswitchd dd4986364575 172.16.31.130:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_db a6692acea93f 172.16.31.130:4000/99cloud/centos-source-nova-novncproxy:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_novncproxy cd704f0bc2c7 172.16.31.130:4000/99cloud/centos-source-nova-consoleauth:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_consoleauth 105978eda1a9 172.16.31.130:4000/99cloud/centos-source-nova-conductor:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_conductor 9d9e7571d7dd 172.16.31.130:4000/99cloud/centos-source-nova-scheduler:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_scheduler 7436ca003817 172.16.31.130:4000/99cloud/centos-source-nova-api:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_api ae8d830091d3 172.16.31.130:4000/99cloud/centos-source-nova-placement-api:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours placement_api f053087cfde5 172.16.31.130:4000/99cloud/centos-source-cinder-scheduler:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_scheduler 8c849182ee63 172.16.31.130:4000/99cloud/centos-source-cinder-api:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_api 129996bd76ee 172.16.31.130:4000/99cloud/centos-source-glance-registry:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours glance_registry fcc184936be5 172.16.31.130:4000/99cloud/centos-source-glance-api:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours glance_api 06a35ba7eb7f 172.16.31.130:4000/99cloud/centos-source-keystone:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours keystone 6821886162bc 172.16.31.130:4000/99cloud/centos-source-rabbitmq:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours rabbitmq 04b556da8365 172.16.31.130:4000/99cloud/centos-source-mariadb:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours mariadb 859a1d8d0137 172.16.31.130:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cron 4e7230af5339 172.16.31.130:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours kolla_toolbox 50be20c91704 172.16.31.130:4000/99cloud/centos-source-fluentd:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours fluentd 72b2538e1beb 172.16.31.130:4000/99cloud/centos-source-memcached:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours memcached compute節點： [root@compute ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0f954ef9095e 172.16.31.130:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_openvswitch_agent 8f62c376e2b5 172.16.31.130:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_vswitchd 005c3dd582a9 172.16.31.130:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_db d6a9b676c0ad 172.16.31.130:4000/99cloud/centos-source-nova-compute:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_compute fbc66c3f40ba 172.16.31.130:4000/99cloud/centos-source-nova-libvirt:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours nova_libvirt 6f942f376c83 172.16.31.130:4000/99cloud/centos-source-nova-ssh:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours nova_ssh 4872cfb2b025 172.16.31.130:4000/99cloud/centos-source-iscsid:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours iscsid c55ec16b0752 172.16.31.130:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cron 426d77add095 172.16.31.130:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours kolla_toolbox 5f959b2372f0 172.16.31.130:4000/99cloud/centos-sourc stroage節點： [root@cinder ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7eca045eeff1 172.16.31.130:4000/99cloud/centos-source-cinder-backup:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_backup bcdf196510f2 172.16.31.130:4000/99cloud/centos-source-cinder-volume:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_volume d59fc8ec7716 172.16.31.130:4000/99cloud/centos-source-tgtd:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours tgtd 7a8d9dab2755 172.16.31.130:4000/99cloud/centos-source-iscsid:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours iscsid ea471af172de 172.16.31.130:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cron 4b41ebf3ea62 172.16.31.130:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours kolla_toolbox 91c086f63421 172.16.31.130:4000/99cloud/centos-source-fluentd:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours fluentd