NOTE:: The below commands are for a 2-node setup (but should apply to other scaled setups too) both running in VMs:
- Controller node:: Nova, Keystone, Cinder, Glance, Neutron (using Open vSwitch plugin and GRE tunneling). - Compute node:: Nova (nova-compute), Neutron (openvswitch-agent)
On Controller node:
$ neutron net-list $ neutron subnet-list $ neutron net $ neutron port-list $ neutron router-list
On Controller node & Compute nodes:
$ ip addr $ route -n $ iptables -L $ cat /etc/sysconfig/iptables $ iptables -nL
On controller node (assuming all Neutron services are running), commands look like:
$ ip netns $ in netns qrouter-UUID ip a $ ip netns qrouter-UUID exec ip link $ ip netns qrouter-UUID exec route -n $ ip netns qrouter-UUID exec iptables -L -t nat $ ip netns qdhcp-UUID ip a $ ip netns qdhcp-UUID ip link $ ip netns qdhcp-UUID exec route -n
Obvious note, Substitute the 'UUID' value accordingly. I omitted it here for brevity.
On both Controller & Compute nodes:
$ ovs-vsctl show $ ovs-dpctl show $ ovs-dpctl dump-flows $ ovs-ofctl dump-flows br-tun $ ovs-ofctl dump-flows br-tun table=21 $ ovs-ofctl dump-flows br-int
On various network devices in play:: OVS bridges, linux bridges, tap devices and veth pairs on compute host, qr, qg interfaces inside network namespaces, physical interfaces, etc.
Some sample commands on various interfaces (not all corner cases are included):
$ tcpdump -envi eth0 | grep -i gre # Run tcpdump on physical link used by GRE tunnels (on Controller # node). This may isolate the problem to the compute node or the # network node. $ tcpdump -i eth0 -n ip proto gre $ tcpdump -envi br-int $ tcpdump -envi br-tun $ ip netns exec qrouter-UUID tcpdump -nn -i qr-63ea2815-b5 icmp $ ip netns exec qrouter-UUID tcpdump -nn -i qg-e7110dba-a9 icmp $ tcpdump -envi 192.168.122.163 $ tcpdump -envi br-ex $ tcpdump -i eth0 -n arp or icmp $ tcpdump -i eth0 -ne ip proto 47 $ tcpdump -i br-ex -n icmp $ tcpdump -i eth0 -n icmp $ tcpdump -i any -n icmp $ tcpdump -i tape7110dba-a9 -n icmp $ tcpdump -envi qvbb71536f2-dd -n arp or icmp $ tcpdump -i eth0 -n not port 22 $ tcpdump -i eth0 -n not port 22 and not port amqp
On physical host:
# Check traffic from Compute host (which is running on # 192.169.142.49) $ tcpdump -nn -i virbr1 host 192.169.142.49