I am working on a setup of OpenStack using LXC containers for virtualization. I can launch instances (at least the instance log reports so). However, I can neither ping them nor connect to them via ssh and the instance log reports that the network interfaces could not be raised properly (see log below). No further errors are raised in the nova and neutron logs.
I tested the LXC setup locally starting the container directly on the compute node and connect to it as intended. For this purpose traffic is bridged directly to the container like here. From this observation I would conclude that all LXC-related module requirements are met (lxc-checkconfig reports so, too). I then extracted the image (tutorial) and provided it to OpenStack for container instantiation.
I, moreover, tried to track the packages transmitted during a ping attempt by tcpdump. They seem to reach the control node but are not relayed to the compute node (or the instance/ container). However, I do not get where they are actually lost and why.
Question:
How I can I find out where the actual problem is , i.e. if it is OpenStack or if it is the networking within the container? I have a bit of a chicken-and-egg problem here as I cannot login to the container to find out if networking there is faulty because I have network issues...
Background:
The OpenStack setup consists of a control node where all scheduling and network services are running and (for now) one compute node where the containers are launched.
The setup is running under Ubuntu 16.04 with a custom built kernel (4.8) on the compute node and uses OpenStack Newton installed via devstack.
Configuration:
devstack configuration on the control node:
[[local|localrc]]
MULTI_HOST=1
RECLONE=no
PIP_UPGRADE=False
PASSWORD=ostack
ADMIN_PASSWORD=$PASSWORD
SERVICE_TOKEN=$PASSWORD
MYSQL_PASSWORD=$PASSWORD
RABBIT_PASSWORD=$PASSWORD
SERVICE_PASSWORD=$PASSWORD
HOST_IP=192.168.0.112
DEST=/opt/stack
SERVICE_DIR=$DEST/status
DATA_DIR=$DEST/data
LOGDIR=$DEST/logs
LOGFILE=$LOGDIR/stack.sh.log
GIT_BASE=http://github.com
export ENABLE_DEBUG_LOG_LEVEL=False
# enable Neutron networking
disable_service n-net
enable_service q-svc q-agt q-dhcp q-l3 q-meta neutron
# Install the tempest test suite
enable_service tempest
disable_service zookeeper
# enable Heat service
enable_service heat h-api h-api-cfn h-api-cw h-eng
# KVM
LIBVIRT_TYPE=qemu
PUBLIC_NETWORK_GATEWAY=192.168.0.1
PUBLIC_INTERFACE=enp4s0
# Neutron configuration
#FIXED_RANGE=10.0.0.0/24
FLOATING_RANGE=192.168.0.0/24
Q_USE_SECGROUP=True
Q_FLOATING_ALLOCATION_POOL=start=192.168.0.5,end=192.168.0.99
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_PHYSICAL_BRIDGE=br-ex
PUBLIC_BRIDGE=br-ex
OVS_BRIDGE_MAPPINGS=public:br-ex
Q_L3_ENABLED=True
devstack configuration on the compute node:
[[local|localrc]]
HOST_IP=192.168.0.106
#
IFACE=br0
#
FLAT_INTERFACE=$IFACE
FIXED_RANGE=10.0.0.0/20
FIXED_NETWORK_SIZE=256
HOST=192.168.0.112
SERVICE_HOST=$HOST
MYSQL_HOST=$HOST
RABBIT_HOST=$HOST
Q_HOST=$HOST
GLANCE_HOSTPORT=$HOST:9292
ADMIN_PASSWORD=ostack
DATABASE_PASSWORDD=$ADMIN_PASSWORD
MYSQL_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
## Neutron options
PUBLIC_INTERFACE=$IFACE
ENABLED_SERVICES=n-cpu,rabbit,q-agt
# newly added
MULTI_HOST=1
DEST=/opt/stack
SERVICE_DIR=$DEST/status
DATA_DIR=$DEST/data
LOGDIR=$DEST/logs
LOGFILE=$LOGDIR/stack.sh.log
GIT_BASE=http://github.com
# nova -- LXC
LIBVIRT_TYPE=lxc
# logging
export ENABLE_DEBUG_LOG_LEVEL=False
lxc configuration on the compute node (/etc/lxc/default.conf):
lxc.network.type = veth
lxc.network.link = br0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:xx:xx:xx
