Guide for setting up multi-node environments with DevStack
==========================================================

Setup
-----

Prepare the Controller (which also runs the Compute service) and Compute
VMs:

    $ virt-builder fedora-24 -o controller.qcow2 --format qcow2 \
        --size 70G  --update --root-password password:testpwd

    $ virt-builder fedora-24 -o compute.qcow2 --format qcow2 \
        --size 70G  --update --root-password password:testpwd


Import the VMs into libvirt (and get access to serial console)

    $ virt-install --name controller --ram 8192 --os-variant fedora23 \
        --disk path=./controller.qcow2,format=qcow2 --nographics --import

    $ virt-install --name compute --ram 6144 --os-variant fedora23 \
        --disk path=./compute.qcow2,format=qcow2 --nographics --import


Configure nested virt on host
-----------------------------

Reference:

http://docs.openstack.org/developer/devstack/guides/devstack-with-nested-kvm.html

Instead of manually editing guest XML one could use the 'virt-xml' tool
to expose virtualization extensions to both the guests:

    $ virt-xml controller --edit --cpu host-passthrough,clearxml=yes

    $ virt-xml compute --edit --cpu host-passthrough,clearxml=yes

The above needs guest reboot to take effect.  (After reboot, check for
the presence of /dev/kvm).


Configure DevStack with multi-node
----------------------------------

Configure Nova, Neutron, Glance and Keystone only services.

Example configs (change IP addresses, etc):

https://kashyapc.fedorapeople.org/virt/openstack/multi-node-configs/

Services on control node:

    ENABLED_SERVICES=g-api,g-reg,key,n-api,n-cpu,n-sch,n-cond,mysql,rabbit,dstat,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta

Service enabled on Compute node:

    ENABLED_SERVICES=n-cpu,rabbit,quantum,q-agt

    * * *

Some caveats while testing live migration with DevStack
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Add this iptables rules on Controller node, so that Compute node can
  reach the RabbitMQ server:
  
    $ iptables -I INPUT -p tcp -m tcp --dport 5672 -j ACCEPT
    $ service iptables save
    $ iptables-save | grep 5672
      -A INPUT -p tcp -m tcp --dport 5672 -j ACCEPT

    $ sudo iptables -I INPUT -p tcp -m tcp --dport 5000 -j ACCEPT                                                                                                       
    $ sudo iptables -I INPUT -p tcp -m tcp --dport 8774 -j ACCEPT 

- Without the below rule, live migration seem to fail with this
  "ConnectionRefused: Unable to establish connection to
  http://compute-node:35357/v2.0/tokens" in n-cpu.log of both, #
  Controller & Compute node.

    $ sudo iptables -I INPUT -p tcp -m tcp --dport 35357 -j ACCEPT

- Migration also fails without these rules:

    $ sudo iptables -I INPUT -p tcp -m tcp --dport 9696 -j ACCEPT
    $ sudo iptables -I INPUT -p tcp -m tcp --dport 9292 -j ACCEPT


Quick check for the presence of the above rules:

    $ egrep -i '5000|8774|35357|9696|9292|5672' /etc/sysconfig/iptables.save