Jump to: navigation, search

OpenDaylight OpenFlow Plugin: TLS Support

Known Issues

CKR_DOMAIN_PARAMS_INVALID exception

I used this as a lead to a workaround: https://bugzilla.redhat.com/show_bug.cgi?id=1022017

On your controller host:

cd /usr/lib/jvm
find .  -name java.security -print
sudo vi {whatever the path is on your system}/java.security

# search for "pkcs11" and
# find a like:

"security.provider.10=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/nss.cfg"

# and comment it out:

#security.provider.10=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/nss.cfg

Overview

A secure controller network will require that both the switches & controllers authenticate to avoid a rogue switch on the controller or a rogue controller driving a switch.

Achieving secure connections between ODL controller and Openflow switches requires management of a PKI ( Private Key Infrastructure ). Openssl provides the tools for this. (If you are planning a production deployment and don't have expertise in TLS PKI, here is a great tutorial which takes you from basic to advanced PKI management: pki-tutorial.pdf )

To get started hacking today with mininet, here we setup a basic PKI and create the necessary certificates to test the controller's openflow plugin with ovs mininet. Note that I based this on the following link: SSL-on-Open-vSwitch-and-ovs-controller.

Create & Sign private/public key certificates

A self-signed "root" CA is created easily with existing mininet tools which leverage openssl, and so will be the starting point for this process. To be secure, the private keys must be protected, while the public key can be freely shared. Because we're using the mininet host also as the CA signing host, the controllers private key exists there, but in production, the key generation for the controller would be isolated from the switches, and only the public controller key would be shared with the switches.

ovs-pki command to generate key pem files

On the mininet host, verify that it's PKI is initialized:

ls /var/lib/openvswitch/pki/controllerca/cacert.pem

If not found, "ovs-pki init" will need to be run as per man pages.

Now run the ovs-pki to create both sets of keys, a controller set and switch set.

cd /etc/openvswitch
sudo ovs-pki req+sign sc switch
sudo ovs-pki req+sign ctl controller

This should have generated six ".pem" files, of which we'll use four. A private key for each side, signed by the unofficial root certificate in /var/lib/openvswitch/pki/. A public certificate for each side. There are also the two request certificates sc-req.pem & ctl-req.pem which were used to generate the signed certificates.

$ ls /etc/openvswitch
conf.db
ctl-cert.pem
ctl-privkey.pem
ctl-req.pem
sc-cert.pem
sc-privkey.pem
sc-req.pem
system-id.conf

Intermediate PKCS12 controller keystore ( to be loaded into java keystores )

Next we'll use openssl to create a keystore in PKCS12 format containing the controller's private and public keys. We'll then import into a JKS format keystore for the ODL controller. Note that it may be possible to use the PKCS12 keystore directly, but I haven't yet tested this.

sudo openssl pkcs12 -export -in ctl-cert.pem -inkey ctl-privkey.pem \
-out ctl.p12 -name odlserver \
-CAfile /var/lib/openvswitch/pki/controllerca/cacert.pem -caname root -chain

You'll be prompted for a password, use "opendaylight"

Enter Export Password:
Verifying - Enter Export Password:

Result is new file ctl.p12:

ls -1 ctl*
ctl-cert.pem
ctl.p12
ctl-privkey.pem
ctl-req.pem

Create Controller keystore ( ctl.jks ) from ctl.p12

Copy two files, ctl.p12 & sc-cert.pem from the mininet host to the ODL host in any work directory. Import the PKCS12 format store into the JKS store consumable by ODL:

# copy these files to mininet's home
sudo cp ctl.p12 sc-cert.pem ~mininet/
# go home (as mininet user)
cd 
# change ownership of those files
sudo chown mininet:mininet ctl.p12 sc-cert.pem

Now use your favorite file transport tool to get these files (scp, sftp, winscp...)

sftp mininet@mininetipaddress
mininet
sftp get ctl.p12 sc-cert.pem
quit

Find keytool in a jdk bin directory and add it to your path for convenience in the future steps. Mine happens to be at /usr/java/jdk1.7.0_51/bin/keytool.

keytool -importkeystore \
        -deststorepass opendaylight -destkeypass opendaylight -destkeystore ctl.jks \
        -srckeystore ctl.p12 -srcstoretype PKCS12 -srcstorepass opendaylight \
        -alias odlserver

Store the switch's public key in a "truststore" ( truststore.jks)

keytool -importcert -file sc-cert.pem -keystore truststore.jks -storepass opendaylight

# when prompted "Trust this certificate? [no]:" enter  "yes"
# Certificate was added to keystore

Finally copy these two keystores to the ssl configuration directory.

<ODLINSTALL> = root folder of karaf distribution

mkdir <ODLINSTALL>/configuration/ssl
cp ctl.jks truststore.jks <ODLINSTALL>/configuration/ssl

Configure ODL's openflow plugin

Note that turning on TLS means only TLS connections are supported ... the connection is "secure."
To use TLS/SSL connections, modify the "42-openflowplugin.xml" or "42-openflowplugin-new.xml" file.

By default this file resides in openflowplugin (OFP) project and

  • if you build OFP then your file is here:
Beryllium: <OPENFLOWPLUGIN_ROOT>/openflowplugin-controller-config/src/main/resources/initial/42-...xml
Boron: <OPENFLOWPLUGIN_ROOT>/openflowplugin-blueprint-config/src/main/resources/initial/*-openflow-connection-config.xml
Carbon: <OPENFLOWJAVA_ROOT>/openflowjava-blueprint-config/src/main/resources/initial/*-openflow-connection-config.xml
  • if you build completely different project with it's own distribution then you need to touch this config file inside living karaf
Beryllium: <ODLINSTALL>/etc/opendaylight/karaf/42-...xml (this file is created upon first run when this feature get installed)
Boron: <ODLINSTALL>/etc/opendaylight/datastore/initial/config/*-openflow-connection-config.xml (this file is created upon first run when this feature get installed)
Carbon: <ODLINSTALL>/etc/opendaylight/datastore/initial/config/*-openflow-connection-config.xml (this file is created upon first run when this feature get installed)

Adapt your config file following way (add/replace green part) for each of existing OF-switch-connection-provider module blocks:

       <module>
         <type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:openflow:switch:connection:provider:impl">prefix:openflow-switch-connection-provider-impl</type>
         <name>openflow-switch-connection-provider-default-impl</name>
         <port>6633</port>
         <switch-idle-timeout>15000</switch-idle-timeout>
         <transport-protocol>TLS</transport-protocol>
         <tls>
           <keystore>configuration/ssl/ctl.jks</keystore>
           <keystore-type>JKS</keystore-type>
           <keystore-path-type>PATH</keystore-path-type>
           <keystore-password>opendaylight</keystore-password>
           <truststore>configuration/ssl/truststore.jks</truststore>
           <truststore-type>JKS</truststore-type>
           <truststore-path-type>PATH</truststore-path-type>
           <truststore-password>opendaylight</truststore-password>
           <certificate-password>opendaylight</certificate-password>
         </tls>

       </module>
       <module>
         <type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:openflow:switch:connection:provider:impl">prefix:openflow-switch-connection-provider-impl</type>
         <name>openflow-switch-connection-provider-legacy-impl</name>
         <port>6653</port>
         <switch-idle-timeout>15000</switch-idle-timeout>
         <transport-protocol>TLS</transport-protocol>
         <tls>
           <keystore>configuration/ssl/ctl.jks</keystore>
           <keystore-type>JKS</keystore-type>
           <keystore-path-type>PATH</keystore-path-type>
           <keystore-password>opendaylight</keystore-password>
           <truststore>configuration/ssl/truststore.jks</truststore>
           <truststore-type>JKS</truststore-type>
           <truststore-path-type>PATH</truststore-path-type>
           <truststore-password>opendaylight</truststore-password>
           <certificate-password>opendaylight</certificate-password>
         </tls>

       </module>


NOTE: Additionally, if you wish to restrict the cipher suites being used for a TLS connection, the desired cipher suite names need to be configured in the config file. For example: add/replace blue part for each of existing OF-switch-connection-provider module blocks.

       <module>
         <type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:openflow:switch:connection:provider:impl">prefix:openflow-switch-connection-provider-impl</type>
         <name>openflow-switch-connection-provider-default-impl</name>
         <port>6633</port>
         <switch-idle-timeout>15000</switch-idle-timeout>
         <transport-protocol>TLS</transport-protocol>
         <tls>
           <keystore>configuration/ssl/ctl.jks</keystore>
           <keystore-type>JKS</keystore-type>
           <keystore-path-type>PATH</keystore-path-type>
           <keystore-password>opendaylight</keystore-password>
           <truststore>configuration/ssl/truststore.jks</truststore>
           <truststore-type>JKS</truststore-type>
           <truststore-path-type>PATH</truststore-path-type>
           <truststore-password>opendaylight</truststore-password>
           <certificate-password>opendaylight</certificate-password>
           <cipher-suites>TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384</cipher-suites>
           <cipher-suites>TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384</cipher-suites> 
           <cipher-suites>TLS_DHE_DSS_WITH_AES_256_GCM_SHA384</cipher-suites>
           <cipher-suites>TLS_DHE_RSA_WITH_AES_256_GCM_SHA384</cipher-suites>
         </tls>

       </module>
       <module>
         <type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:openflow:switch:connection:provider:impl">prefix:openflow-switch-connection-provider-impl</type>
         <name>openflow-switch-connection-provider-legacy-impl</name>
         <port>6653</port>
         <switch-idle-timeout>15000</switch-idle-timeout>
         <transport-protocol>TLS</transport-protocol>
         <tls>
           <keystore>configuration/ssl/ctl.jks</keystore>
           <keystore-type>JKS</keystore-type>
           <keystore-path-type>PATH</keystore-path-type>
           <keystore-password>opendaylight</keystore-password>
           <truststore>configuration/ssl/truststore.jks</truststore>
           <truststore-type>JKS</truststore-type>
           <truststore-path-type>PATH</truststore-path-type>
           <truststore-password>opendaylight</truststore-password>
           <certificate-password>opendaylight</certificate-password>
           <cipher-suites>TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384</cipher-suites>
           <cipher-suites>TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384</cipher-suites> 
           <cipher-suites>TLS_DHE_DSS_WITH_AES_256_GCM_SHA384</cipher-suites>
           <cipher-suites>TLS_DHE_RSA_WITH_AES_256_GCM_SHA384</cipher-suites>
         </tls>

       </module>

To restrict cipher suites used, the Java Cryptography Extension policy files should be updated to include jars that provide unlimited cryptographic strength. Follow the README files at the below links to install the jar files: For JDK8: http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html For JDK7: http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html

Post adaptation note:

touched <OPENFLOWPLUGIN_ROOT>
make sure you REBUILD openflowplugin project using maven clean install at the root directory. (your change is bound to source files of openflowplugin)
touched <ODLINSTALL>
make sure you DO NOT REBUILD your karaf distribution project otherwise the change will get overwritten. (your change is bound to current karaf build ONLY!)

Exemplary configuration

There is already exemplary code in configuration/initial/42-openflowplugin.xml file and also exemplary keys stored in openflowjava (src/main/resources). This exemplary code is commented, so the default is to use unsecured communication.

If you want to try TLS secured communication with your device, you need to do following steps:

  • make sure that <transport-protocol> is set with TLS
  • uncomment code in <tls> tags
  • find exemplary-* files in openflowjava repository - under openflow-protocol-impl/src/main/resources
  • copy exemplary-switch-privkey.pem, exemplary-switch-cert.pem and exemplary-cacert.pem files into your device
  • configure your device with provided keys (in case of openvswitch please see "Configure openvswitch SSL" part below)
  • start communication

Now you should be able to communicate over TLS.

Configure openvswitch SSL

set ovs ssl options

sudo ovs-vsctl set-ssl \
    /etc/openvswitch/sc-privkey.pem \
    /etc/openvswitch/sc-cert.pem \
    /var/lib/openvswitch/pki/controllerca/cacert.pem

Start a mininet with SSL connections to the ODL controller

open a file "ssl_switch_tests.py"


#!/usr/bin/python
from mininet.net import Mininet
from mininet.node import Controller, RemoteController
from mininet.cli import CLI
from mininet.log import setLogLevel, info

def emptyNet():
    net = Mininet(controller=None)
    net.addController( 'c0', controller=RemoteController, ip='YOUR_CONTROLLER_IP', port=6633)
    h1 = net.addHost( 'h1' )
    h2 = net.addHost( 'h2' )
    s1 = net.addSwitch( 's1' )
    net.addLink( h1, s1 )
    net.addLink( h2, s1 )

    net.start()
    s1.cmd('ovs-vsctl set-controller s1 ssl:YOUR_CONTROLLER_IP:6633')

    CLI( net )
    net.stop()

if __name__ == '__main__':
    setLogLevel( 'info' )
    emptyNet()

Start mininet :

chmod +x ssl_switch_test.py
sudo ./ssl_switch_test.py

Example Hardware Switch Configuration

Brocade MLX

After setting up a tftp server, copy sc-cert.pem and sc-privkey.pem into the proper upload location: In this example, there is a tftp server running on the controller host "10.0.0.1"

telnet@NetIron MLX-4 Router#enable
<enter config password>
telnet@NetIron MLX-4 Router(config)#copy tftp flash 10.0.0.1 sc-cert.pem client-certificate
telnet@NetIron MLX-4 Router(config)#copy tftp flash 10.0.0.1 sc-privkey.pem client-private-key
telnet@NetIron MLX-4 Router(config)#openflow controller ip-address 10.0.0.1

Debugging

mininet debugging

You'll see connection entries in the ovswitchd log file:

sudo tail /var/log/openvswitch/ovs-vswitchd.log

ODL controller debugging

./karaf  -Djavax.net.debug=ssl,handshake

SSL23_GET_SERVER_HELLO:unknown protocol

If you have errors, try to create a SSL connection between the hosts, to verify if your certificates are valid.

Server

 sudo openssl s_server -accept 7569 -cert /etc/openvswitch/sc-cert.pem -CAfile /var/lib/openvswitch/pki/controllerca/cacert.pem -key /etc/openvswitch/sc-privkey.pem 


Client

sudo openssl s_client -showcerts -connect HOST_IP:7569 -CAfile sc-cert.pem


Set openflowjava log level to TRACE and verify messages

opendaylight-user@root> set TRACE org.opendaylight.openflowjava