Set-up: A 2-node RDO setup (using Fedora 20, and IceHouse packages from Fedora Rawhide as of this writing 14-MAY-2014) with Nova, Keystone, Glance and Neutron with ML2+GRE+OVS. RabbitMQ for AMQP messaging. This is a virtualized setup, i.e. Nova instances are KVM nested guests. Below are the configuration files for Neutron and Nova on both Controller (192.169.142.97) and Compute (192.169.142.168) nodes. And, iptables rules. Neutron configurations ---------------------- Neutron Controller node ~~~~~~~~~~~~~~~~~~~~~~~ 1. neutron.conf: $ cat /etc/neutron/neutron.conf | grep -v ^$ | grep -v ^# [DEFAULT] verbose = True debug = True agent_down_time = 75 lock_path = $state_path/lock notification_driver = neutron.openstack.common.notifier.rpc_notifier auth_strategy = keystone rpc_backend = neutron.openstack.common.rpc.impl_kombu rabbit_host = 192.169.142.97 rabbit_userid = guest rabbit_password = fedora notify_nova_on_port_status_changes = True notify_nova_on_port_data_changes = True nova_url = http://192.169.142.97:8774/v2 nova_admin_username = nova nova_admin_tenant_id = 3e112abc4c4b4214b8efbd627a32f75e nova_admin_password = fedora nova_admin_auth_url = http://192.169.142.97:35357/v2.0 core_plugin = ml2 service_plugins = router [quotas] [agent] report_interval = 5 [keystone_authtoken] auth_host = 192.169.142.97 auth_port = 35357 auth_protocol = http admin_tenant_name = services admin_user = neutron admin_password = fedora signing_dir = $state_path/keystone-signing auth_uri = http://192.169.142.97:5000 [database] connection = mysql://neutron:fedora@192.169.142.97/neutron [service_providers] service_provider=VPN:openswan:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default 2. ml2_conf.ini: $ cat /etc/neutron/plugins/ml2/ml2_conf.ini | grep -v ^$ | grep -v ^# [ml2] type_drivers = gre tenant_network_types = gre mechanism_drivers = openvswitch [ml2_type_flat] [ml2_type_vlan] [ml2_type_gre] tunnel_id_ranges = 1:1000 [ml2_type_vxlan] [securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver enable_security_group = True [ovs] local_ip = 192.169.142.97 [agent] tunnel_types = gre root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf 3. ovs_neutron_plugin.ini (same as ml2_config.ini): $ cat /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini | grep -v ^$ | grep -v [ml2] type_drivers = gre tenant_network_types = gre mechanism_drivers = openvswitch [ml2_type_flat] [ml2_type_vlan] [ml2_type_gre] tunnel_id_ranges = 1:1000 [ml2_type_vxlan] [securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver enable_security_group = True [ovs] local_ip = 192.169.142.97 [agent] tunnel_types = gre root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf 4. dhcp_agent.ini: $ cat /etc/neutron/dhcp_agent.ini | grep -v ^$ | grep -v ^# [DEFAULT] debug = False resync_interval = 30 interface_driver =neutron.agent.linux.interface.OVSInterfaceDriver dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq external_network_bridge = br-ex use_namespaces = True root_helper=sudo neutron-rootwrap /etc/neutron/rootwrap.conf state_path=/var/lib/neutron dnsmasq_config_file = /etc/neutron/dnsmasq.conf 5. l3_agent.ini: $ cat /etc/neutron/l3_agent.ini | grep -v ^$ | grep -v ^# [DEFAULT] debug = False resync_interval = 30 interface_driver =neutron.agent.linux.interface.OVSInterfaceDriver dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq external_network_bridge = br-ex use_namespaces = True state_path=/var/lib/neutron metadata_ip = 192.169.142.97 metadata_port = 8755 6. dnsmasq.conf: $ cat /etc/neutron/dnsmasq.conf | grep -v ^$ | grep -v ^# log-facility = /var/log/neutron/dnsmasq.log log-dhcp 7. metadata_agent.ini: $ cat metadata_agent.ini | grep -v ^$ | grep -v ^# [DEFAULT] auth_url = http://192.169.142.97:35357/v2.0/ auth_region = regionOne admin_tenant_name = services admin_user = neutron admin_password = fedora nova_metadata_ip = 192.169.142.97 nova_metadata_port = 8755 metadata_proxy_shared_secret = fedora Neutron Compute node ~~~~~~~~~~~~~~~~~~~~ 1. neutron.conf: $ cat /etc/neutron/neutron.conf | grep -v ^$ | grep -v ^# [DEFAULT] verbose = True debug = True agent_down_time = 75 lock_path = $state_path/lock notification_driver = neutron.openstack.common.notifier.rpc_notifier auth_strategy = keystone rpc_backend = neutron.openstack.common.rpc.impl_kombu rabbit_host = 192.169.142.97 rabbit_userid = guest rabbit_password = fedora notify_nova_on_port_status_changes = True notify_nova_on_port_data_changes = True nova_url = http://192.169.142.97:8774/v2 nova_admin_username = nova nova_admin_tenant_id = 3e112abc4c4b4214b8efbd627a32f75e nova_admin_password = fedora nova_admin_auth_url = http://192.169.142.97:35357/v2.0 core_plugin = ml2 service_plugins = router [quotas] [agent] report_interval = 5 [keystone_authtoken] auth_host = 192.169.142.97 auth_port = 35357 auth_protocol = http admin_tenant_name = services admin_user = neutron admin_password = fedora signing_dir = $state_path/keystone-signing auth_uri = http://192.169.142.97:5000 [database] connection = mysql://neutron:fedora@192.169.142.97/neutron [service_providers] service_provider=VPN:openswan:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default 2. ml2_conf.ini: $ cat plugins/ml2/ml2_conf.ini | grep -v ^$ | grep -v ^# [ml2] type_drivers = gre tenant_network_types = gre mechanism_drivers = openvswitch [ml2_type_flat] [ml2_type_vlan] [ml2_type_gre] tunnel_id_ranges = 1:1000 [ml2_type_vxlan] [securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver enable_security_group = True [ovs] local_ip = 192.169.142.168 [agent] tunnel_types = gre root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf NOTES: - ovs_neutron_plugin.ini is same as ml2_conf.ini - metadata_agent.ini is same on both nodes Nova configurations ------------------- Nova Compute node ----------------- 1. nova.conf $ cat /etc/nova/nova.conf | grep -v ^$ | grep -v ^# [DEFAULT] logdir = /var/log/nova verbose = True debug = True state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_use_virtio_for_bridges=True libvirt_inject_partition = -1 vif_plugging_is_fatal = False vif_plugging_timeout = 0 sql_connection = mysql://nova:nova@192.169.142.97/nova compute_driver = libvirt.LibvirtDriver libvirt_type=kvm rootwrap_config = /etc/nova/rootwrap.conf auth_strategy = keystone firewall_driver=nova.virt.firewall.NoopFirewallDriver enabled_apis = ec2,osapi_compute,metadata my_ip=192.169.142.97 network_api_class = nova.network.neutronv2.api.API neutron_url = http://192.169.142.97:9696 neutron_auth_strategy = keystone neutron_admin_tenant_name = services neutron_admin_username = neutron neutron_admin_password = fedora neutron_admin_auth_url = http://192.169.142.97:35357/v2.0 linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver firewall_driver = nova.virt.firewall.NoopFirewallDriver security_group_api = neutron rpc_backend = nova.rpc.impl_kombu rabbit_host = 192.169.142.97 rabbit_port = 5672 rabbit_userid = guest rabbit_password = fedora glance_host = 192.169.142.97 metadata_host = 192.169.142.97 metadata_listen = 0.0.0.0 metadata_listen_port = 8755 service_neutron_metadata_proxy = True neutron_metadata_proxy_shared_secret = fedora [keystone_authtoken] auth_uri = http://192.169.142.97:5000 admin_tenant_name = services admin_user = nova admin_password = fedora auth_host = 192.169.142.97 auth_port = 35357 auth_protocol = http signing_dirname = /tmp/keystone-signing-nova Nova Compute node ~~~~~~~~~~~~~~~~~ 1. nova.conf: $ cat nova.conf | grep -v ^$ | grep -v ^# [DEFAULT] logdir = /var/log/nova state_path = /var/lib/nova lock_path = /var/lib/nova/tmp volumes_dir = /etc/nova/volumes dhcpbridge = /usr/bin/nova-dhcpbridge dhcpbridge_flagfile = /etc/nova/nova.conf force_dhcp_release = True injected_network_template = /usr/share/nova/interfaces.template libvirt_nonblocking = True libvirt_use_virtio_for_bridges=True libvirt_inject_partition = -1 sql_connection = mysql://nova:nova@192.169.142.97/nova compute_driver = libvirt.LibvirtDriver libvirt_type=qemu rootwrap_config = /etc/nova/rootwrap.conf auth_strategy = keystone firewall_driver=nova.virt.firewall.NoopFirewallDriver enabled_apis = ec2,osapi_compute,metadata my_ip=192.169.142.168 network_api_class = nova.network.neutronv2.api.API neutron_url = http://192.169.142.97:9696 neutron_auth_strategy = keystone neutron_admin_tenant_name = services neutron_admin_username = neutron neutron_admin_password = fedora neutron_admin_auth_url = http://192.169.142.97:35357/v2.0 linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver firewall_driver = nova.virt.firewall.NoopFirewallDriver security_group_api = neutron rpc_backend = nova.rpc.impl_kombu rabbit_host = 192.169.142.97 rabbit_port = 5672 rabbit_userid = guest rabbit_password = fedora glance_host = 192.169.142.97 [keystone_authtoken] auth_uri = http://192.169.142.97:5000 admin_tenant_name = services admin_user = nova admin_password = fedora auth_host = 192.169.142.97 auth_port = 35357 auth_protocol = http signing_dirname = /tmp/keystone-signing-nova iptables -------- iptables on Controller ~~~~~~~~~~~~~~~~~~~~~~ $ cat /etc/sysconfig/iptables *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -s 192.169.142.97/32 -p tcp -m multiport --dports 5671,5672 -m comment --comment "001 amqp incoming amqp_192.169.142.97" -j ACCEPT -A INPUT -p tcp -m multiport --dports 3260 -m comment --comment "001 cinder incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 80 -m comment --comment "001 horizon incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9292 -m comment --comment "001 glance incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5000,35357 -m comment --comment "001 keystone incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 3306 -m comment --comment "001 mariadb incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 6080 -m comment --comment "001 novncproxy incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8770:8780 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9696 -m comment --comment "001 neutron incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5672 -m comment --comment "001 qpid incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8700 -m comment --comment "001 metadata incoming" -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5999 -j ACCEPT -A INPUT -j REJECT --reject-with icmp-host-prohibited -A INPUT -p gre -j ACCEPT -A OUTPUT -p gre -j ACCEPT -A FORWARD -j REJECT --reject-with icmp-host-prohibited COMMIT iptables on Compute node ~~~~~~~~~~~~~~~~~~~~~~~~ $ cat /etc/sysconfig/iptables *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5999 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT -A INPUT -p gre -j ACCEPT -A INPUT -j REJECT --reject-with icmp-host-prohibited -A OUTPUT -p gre -j ACCEPT -A FORWARD -j REJECT --reject-with icmp-host-prohibited COMMIT