Jump to: navigation, search

NetVirt: Multiple VLAN provider networks HowTo

Overview

This page describes how to enable multiple external VLAN provider networks using opendaylight controller. Below mentioned is the use-case diagram for this network:

Wikipic.png


Installing OpenDaylight

Prerequisites: OpenDaylight dev environment with JDK8

  • Download the latest distro from the below link:
 https://nexus.opendaylight.org/content/sites/site/ 
  • Untar the distribution :
 tar -xvzf distribution-karaf-0.5.0-Boron.tar.gz
  • cd distribution-karaf-0.5.0-Boron
  • Edit the /etc/custom.properties file, add the below mentioned line:
ovsdb.l3.multiple.network.enabled=yes 
  • Run karaf from the distro directory  :
 ./bin/karaf
  • Install ovsdb openstack feature:
 feature:install odl-ovsdb-openstack

Note:Check the logs to make sure there are not any fatal exceptions. Before we proceed make sure ODL is listening on ports 8080, 6640 and 6633.

Installing OpenStack

Installing the Controller node

  • Add external bridge:
 sudo ovs-vsctl add-br br-ex

Note: If the below error occurs while executing the show command:

[root@localhost ~]# sudo ovs-vsctl add-br br-ex
sudo: ovs-vsctl: command not found

Install openvswitch in the machine.

  • Add port to the external bridge.
sudo ovs-vsctl add-port br-ex <VLAN Network port>

Note:If you do not have VLAN network on setup, follow the below steps to create Multiple IP's on single interface with different VLAN make following changes in openstack control node and external VM's.

[root@control-stack ~(keystone_admin)]# cat /etc/sysconfig/network-scripts/ifcfg-ens36
TYPE=Ethernet
BOOTPROTO=static
NM_CONTROLLED=no
NAME=ens36
DEVICE=ens36
ONBOOT=yes

Note: ens36 should have an ip.

[root@control-stack ~(keystone_admin)]# cat /etc/sysconfig/network-scripts/ifcfg-ens36.100
TYPE=Ethernet
DEVICE=ens36.100
ONBOOT=yes
VLAN=yes
BOOTPROTO=none
NETMASK=255.255.255.0
NM_CONTROLLED=no
IPADDR=192.165.6.10

[root@control-stack ~(keystone_admin)]# cat /etc/sysconfig/network-scripts/ifcfg-ens36.101
TYPE=Ethernet
DEVICE=ens36.101
ONBOOT=yes
VLAN=yes
BOOTPROTO=none
NETMASK=255.255.255.0
NM_CONTROLLED=no
IPADDR=192.165.7.10

  • Restart the network service with the command given below:
service network restart
  • After making the above changes, it can be verified by giving "ifconfig" as shown below:
[root@control-stack ~(keystone_admin)]#ifconfig
ens36.100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.165.6.10  netmask 255.255.255.0  broadcast 192.165.6.255
        inet6 fe80::250:56ff:fe84:266  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:84:02:66  txqueuelen 0  (Ethernet)
        RX packets 16  bytes 1020 (1020.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 23  bytes 1670 (1.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens36.101: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.165.7.10  netmask 255.255.255.0  broadcast 192.165.7.255
        inet6 fe80::250:56ff:fe84:266  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:84:02:66  txqueuelen 0  (Ethernet)
        RX packets 628  bytes 51280 (50.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 633  bytes 59938 (58.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  • External bridge and port creation: The bridges and ports have to be added manually as shown below:
sudo ovs-vsctl add-br br-ex
sudo ovs-vsctl add-port br-ex ens36
sudo ovs-vsctl add-port br-ex ens36.100
ovs-vsctl set port ens36.100 tag=100
sudo ovs-vsctl add-port br-ex ens36.101
ovs-vsctl set port ens36.101 tag=101
  • Create the local.conf file as shown below:
[[local|localrc]]
LOGFILE=stack.sh.log
SCREEN_LOGDIR=/opt/stack/data/log
LOG_COLOR=False
#OFFLINE=True
RECLONE=yes

# disable everything so we can explicitly enable only what we need
disable_all_services
# Core compute (glance+keystone+nova+vnc)
enable_service g-api g-reg key n-api n-crt n-obj n-cpu n-cond n-sch n-novnc n-xvnc n-cauth q-agt

# next line enables default l2 agent and not odl
enable_service neutron q-dhcp q-meta q-svc horizon

# additional services
enable_service mysql rabbit

enable_plugin networking-odl https://github.com/openstack/networking-odl stable/mitaka

ENABLE_TENANT_VLANS=True
ENABLE_TENANT_TUNNELS=True
NEUTRON_CREATE_INITIAL_NETWORKS=False
Q_ML2_PLUGIN_MECHANISM_DRIVERS=opendaylight
Q_ML2_PLUGIN_TYPE_DRIVERS=vlan,gre,vxlan,flat
HOST_IP=10.106.138.104
#HOST_NAME=localhost
SERVICE_HOST=$HOST_IP

NEUTRON_CREATE_INITIAL_NETWORKS=False
Q_ML2_TENANT_NETWORK_TYPE=vxlan
ENABLE_TENANT_TUNNELS=True

# https://github.com/openstack/networking-odl/blob/master/devstack/settings
ODL_MODE=externalodl # allinone is the default mode. This line is here just to make that point.

ODL_MGR_IP=10.106.138.210
ODL_PORT=8080
#ODL_BOOT_WAIT=123
#ODL_OVS_MANAGERS=10.106.138.154,10.106.138.155,10.106.138.110

VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP
VNCSERVER_LISTEN=0.0.0.0

DATABASE_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
SERVICE_TOKEN=service
QPID_PASSWORD=rabbit
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Disable q-l3 and uncomment the lines below if ODL is being configured to perform l3fwd
# For more info:  https://github.com/openstack/networking-odl/blob/master/devstack/README.rst
enable_service q-l3
Q_L3_ENABLED=True
#ODL_L3=True
OVS_BRIDGE_MAPPINGS=physnet1:br-ex
ML2_VLAN_RANGES=physnet1:100:101

#L3-agent config to use Multiple external network.
[[post-config|$Q_L3_CONF_FILE]]
[DEFAULT]
external_network_bridge=
router_delete_namespaces=True
gateway_external_network_id=

[[post-config|$NEUTRON_CONF]]
[DEFAULT]
service_plugins = router

[[post-config|/etc/neutron/plugins/ml2/ml2_conf.ini]]
[agent]
minimize_polling=True

[ovs]
local_ip=$HOST_IP

[[post-config|/etc/nova/nova.conf]]
[DEFAULT]
novncproxy_base_url=http://10.106.138.104:8175/vnc_auto.html
novncproxy_port=8175
novncproxy_host = 0.0.0.0
use_ipv6=True
  • Now set the provider mappings:
sudo ovs-vsctl set Open_vSwitch <OVS UUID> other_config:provider_mappings=physnet1:br-ex 
sudo ovs-vsctl set Open_vSwitch <OVS UUID> other_config:local_ip=10.106.138.104

Note: To get the uuid give the below command:

[stack@compute2 devstack]$ sudo ovs-vsctl get Open_vSwitch . _uuid
57c01c8d-c1c7-4fcf-b3c5-c61ad0babc38
  • Then stack the control node:
./stack.sh
  • Check patch ports are created in control node after successful installation.
[root@ODL-packstack ~]# sudo ovs-vsctl show
097a4008-5128-41ae-bfa0-8d0019e8b084
    Manager "tcp:10.106.138.159:6640"
        is_connected: true
    Bridge br-ex
        Port "ens36"
            Interface "ens36"
        Port "ens36.101"
            tag: 101
            Interface "ens36.101"
        Port br-ex
            Interface br-ex
                type: internal
        Port patch-ext
            Interface patch-ext
                type: patch
                options: {peer=patch-int}
        Port "ens36.100"
            tag: 100
            Interface "ens36.100"
    Bridge br-int
        Controller "tcp:10.106.138.159:6653"
            is_connected: true
        fail_mode: secure
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-ext}
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.5.0"

Installing Compute Node

  • Create the local.conf file :
[[local|localrc]]
enable_plugin networking-odl https://github.com/openstack/networking-odl stable/mitaka
LOGFILE=stack.sh.log
LOG_COLOR=False
SCREEN_LOGDIR=/opt/stack/data/log
#OFFLINE=True
RECLONE=yes

disable_all_services
enable_service n-cpu n-novnc n-cauth
enable_service neutron q-agt

HOST_IP=10.106.138.127
SERVICE_HOST=10.106.138.202

VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP
VNCSERVER_LISTEN=0.0.0.0

ODL_MODE=compute
ODL_MGR_IP=10.106.138.159
#ODL_OVS_MANAGERS=10.106.138.138,10.106.138.192,10.106.138.137
ENABLE_TENANT_TUNNELS=True
Q_ML2_TENANT_NETWORK_TYPE=vxlan

Q_HOST=$SERVICE_HOST
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
KEYSTONE_AUTH_HOST=$SERVICE_HOST
KEYSTONE_SERVICE_HOST=$SERVICE_HOST

DATABASE_PASSWORD=mysql
RABBIT_PASSWORD=rabbit
QPID_PASSWORD=rabbit
SERVICE_TOKEN=service
SERVICE_PASSWORD=admin
ADMIN_PASSWORD=admin

# Uncomment lines below if odl-compute is to be used for l3 forwarding
enable_service q-l3
Q_L3_ENABLED=True
#ODL_L3=True
[[post-config|/etc/neutron/plugins/ml2/ml2_conf.ini]]
[agent]
minimize_polling=True

[ovs]
local_ip=$HOST_IP

[[post-config|/etc/nova/nova.conf]]
[DEFAULT]
novncproxy_base_url=http://10.106.138.127:8175/vnc_auto.html
novncproxy_port=8175
novncproxy_host = 0.0.0.0
use_ipv6=True

  • Stack the Compute node
./stack.sh

Configure provider VLAN network and router in control Node

  • For the Provider Network1 please execute the below mentioned commands:
cd devstack
source openrc admin admin
neutron net-create provider-vlan100 --provider:network_type vlan --router:external true --provider:physical_network physnet1 --provider:segmentation_id 100
neutron subnet-create --name subnet-provider-100 provider-vlan100 192.165.6.0/24 --enable_dhcp=False --allocation_pool start=192.165.6.25,end=192.165.6.45 --gateway 192.165.6.17
neutron router-create router1
neutron router-gateway-set router1 provider-vlan100
neutron floatingip-create provider-vlan100
  • For the Provider Network 2
neutron net-create provider-vlan101 --provider:network_type vlan --router:external true --provider:physical_network physnet1 --provider:segmentation_id 101
neutron subnet-create --name subnet-provider-101 provider-vlan101 192.165.7.0/24 --enable_dhcp=False --allocation_pool start=192.165.7.25,end=192.165.7.45 --gateway 192.165.7.17
neutron router-create router2	
neutron router-gateway-set router2 provider-vlan101	
neutron floatingip-create provider-vlan101

Openstack GUI

Configure the Internal Network:

  • create 2 internal network(vxlan) and subnet (Network1 and Network2).

Net1 Creation.jpg Net1-Subnet Creation.jpg Net2 Creation.jpg Net2- Subnet Creation.jpg

  • create VM1 using Network1. (control node)

VM1 Creation(control).jpg VM1 Network Selection.jpg

  • create VM2 using Network2. (compute node)

VM2 creation(compute).jpg VM2 Net Selection.jpg

  • Attach Network1 interface to the router1.

AddInterfaceToRouter1.jpg

  • Attach Network2 interface to the router2.

AddInterfaceToRouter2.jpg

  • Associate floating ip to the created VM

FloatingIPTOVM1.jpg FloatingIPTOVM2.jpg

Network Topology

Network topology.png


Testing reachability

  • Login to the Network1 VM instance try to reach external gateway 192.165.7.17

PingFromInternalVM1.jpg

  • Login to the Network2 VM instance try to reach external gateway 192.165.6.17

PingFromInternalVM2.jpg