Jump to: navigation, search

Controller Core Functionality Tutorials:Tutorials:Data Store Benchmarking and Data Access Patterns


Introduction

The purpose of this tutorial is twofold: to provide basic insights into how the MD-SAL data store operates and performs, and give examples of basic data access patterns that you can use to use in your application. It also includes a discussion and a couple of tips how to design your application for best performance and scale.

The tutorial is structured as an OpenDaylight plugin ("internal" application), It was created with help of the MD-SAL Startup Project Archetype, which was used to create the overall structure of the application and the skeletons for its core files The core skeletons were expanded and modified as needed, and java files that implement the application logic were added. The directory structure is shown below:

.
├── dsbenchmark-api
├── dsbenchmark-artifacts
├── dsbenchmark-features
├── dsbenchmark-impl
├── dsbenchmark-karaf
├── dsbenchmark-scripts
└── pom.xml

The content of the folders is as follows:

  • dsbenchmark-api: contains the definition of the application's service yang model.
  • dsbenchmark-features: contains the definition of the application's features; no modifications to the default artifact settings were made.
  • dsbenchmark-impl: contains all the Java code for the application; the classes generated by default by the archetype and yang tools (DsbenchmarkImplModule.java, DsbenchmarkImpl.java) were modified and new classes implementing the app's functionality were added. The code is structured as a set of simple tests, each showcasing and testing one particular way of using the data store. The files inside the dsbenchmark-impl folder are structured as follows:
   ├── dsbenchmark-impl
   │   ├── pom.xml
   │   └── src
   │       └── main
   │           ├── config
   │           │   ├── default-config.xml
   │           │   └── maven-metadata-local.xml
   │           ├── java
   │           │   └── org
   │           │       └── opendaylight
   │           │           ├── dsbenchmark
   │           │           │   ├── BaListBuilder.java
   │           │           │   ├── DatastoreAbstractWriter.java
   │           │           │   ├── DomListBuilder.java
   │           │           │   ├── DsbenchmarkProvider.java
   │           │           │   ├── simpletx
   │           │           │   │   ├── SimpletxBaDelete.java
   │           │           │   │   ├── SimpletxBaWrite.java
   │           │           │   │   ├── SimpletxDomDelete.java
   │           │           │   │   └── SimpletxDomWrite.java
   │           │           │   └── txchain
   │           │           │       ├── TxchainBaDelete.java
   │           │           │       ├── TxchainBaWrite.java
   │           │           │       ├── TxchainDomDelete.java
   │           │           │       └── TxchainDomWrite.java
  • the dsbenchmark folder contains the framework for the the app: the service, the test functionality, etc. The files in the main folder folder are as follows:
      • BaListBuilder.java: Binding-aware builder for the list defined by the yang model in dsbenchmark-api
      • DatastoreAbstractWriter.java.java: defines the API and functions common for all test classes
      • DomListBuilder.java: DOM (Binding-independent builder for the list defined by the yang model in dsbenchmark-api
      • DsbenchmarkProvider.java: Entry point and framework for the dsbenchmark applicationon
    • simpletx: contains the test/example classes that use synchronous transactions
      • SimpletxBaDelete.java: showing how to use binding-aware DELETE transactions;
      • SimpletxBaWrite.java: showing how to use binding-aware PUT and MERGE transactions;
      • SimpletxDomDelete.java: showing how to use binding-independent DELETE transactions;
      • SimpletxDomWrite.java: showing how to use binding-independent PUT and MERGE transactions;
    • txchain: contains the test/example classes that use transaction chaining
      • TxchainBaDelete.java: showing how to use binding-aware DELETE transactions;
      • TxchainBaWrite.java: showing how to use binding-aware PUT and MERGE transactions;
      • TxchainDomDelete.java: showing how to use binding-independent DELETE transactions;
      • TxchainDomWrite.java: showing how to use binding-independent PUT and MERGE transactions;
  • dsbenchmark'-karaf: used to build the karaf distribution for testing the application. No modifications to the default artifact settings were made
  • dsbenchmark-scripts: contains python scripts that drive the execution of the tests, collect test results and dump them into a .csv file for import in to Excel or another graph-plotting software

You can get the code from the coretutorials git repository:

> git clone https://git.opendaylight.org/gerrit/p/coretutorials.git
> cd coretutorials/dsbenchmark

To run dsbenchmark tests -

> python dsbenchmark.py --help
   usage: dsbenchmark.py [-h] [--host HOST] [--port PORT]
                     [--txtype {TX-CHAINING,SIMPLE-TX} [{TX-CHAINING,SIMPLE-TX} ...]]
                     [--total TOTAL] [--inner INNER] [--ops OPS]
                     [--optype {PUT,MERGE,DELETE} [{PUT,MERGE,DELETE} ...]]
                     [--format {BINDING-AWARE,BINDING-INDEPENDENT} [{BINDING-AWARE,BINDING-INDEPENDENT} ...]]
                     [--warmup WARMUP] [--runs RUNS]
  optional arguments:
 -h, --help            show this help message and exit
 --host HOST         the IP of the target host to initiate benchmark testing on.
 --port PORT         the port number of target host.
 --txtype {TX-CHAINING,SIMPLE-TX} [{TX-CHAINING,SIMPLE-TX} ...] 
                             list of the transaction types to execute.
 --total TOTAL       total number of elements to process.
 --inner INNER       number of inner elements to process.
 --ops OPS            number of operations per transaction.
 --optype {PUT,MERGE,DELETE} [{PUT,MERGE,DELETE} ...]
                             list of the types operations to execute.
 --format {BINDING-AWARE,BINDING-INDEPENDENT} [{BINDING-AWARE,BINDING-INDEPENDENT} ...]
                             list of data formats to execute.
 --warmup WARMUP     number of warmup runs before official test runs
 --runs RUNS         number of official test runs. 
  Note: Reported results are based on these runs.

Background

This tutorial focuses on MD-SAL's data store functionality. There are two data formats supported by the data store: binding aware and binding independent. The Binding-Aware data format allows you to work with Java APIs and Data Transfer Objects (DTOs) generated from your yang models. The Binding Independent data format (a.k.a. DOM Data Format) is the data store's native format (i.e. data in the data store is stored in this format).

The binding-independent data format is supported by a binding-independent DOM Broker, which interprets YANG models at runtime. It is the core component of the MD-SAL runtime. The Binding-Aware data format is supported by a Binding-Aware Broker, which exposes Java APIs for applications/plugins using binding-aware representation of data. More details can be found in MD-SAL Overview.

The Benchmark Application

The Service Yang Model

The apps' service yang model is defined in dsbenchmark-api/src/main/yang/dsbenchmark.yang. It has three main parts:

  • container test-exec in the configuration space that contains test data stored into the md-sal data by the test application; this container also contains the definition of the test data - basically a list of lists. The test data definitions is as follows:
   container test-exec {
       config true;
       list outer-list {
           key id;
           leaf id {
               type int32;
           }
           choice outer-choice {
               case one {
                   leaf one {
                       type string;
                   }
               }
               case two-three {
                   leaf two {
                       type string;
                   }
                   leaf three {
                       type string;
                   }
              }
          }
          list inner-list {
               key name;
               leaf name {
                   type int32;
               }
               leaf value {
                   type string;
               }
           }
       }
   }
  • container test-status contains the operational state of the application:
    • Indicator whether a test is in progress; only one test can run at a time;
    • The number of performed test runs
  • The application'ss "REST API": a set of control rpc services that perform test actions:
    • cleanup-store: deletes all entries the test-exec configuration space
    • start-test; starts a test run with parameters specified in the service's yang model. the RPC returns test status and measurements from the test:
   rpc start-test {
       description
         "Start a new data store write test run";
       input {
           leaf operation {
               mandatory true;
               type enumeration {
                   enum "PUT" {
                       value 1;
                       description
                         "The put operation";
                   }
                   enum "MERGE" {
                       value 2;
                       description
                         "The merge operation";
                   }
                   enum "DELETE" {
                       value 3;
                       description
                           "Delete items from a list sotred in the data store";
                   }
               }
               description
                   "Type of the transaction operation to benchmark";
           }
           leaf data-format {
               mandatory true;
               type enumeration {
                   enum "BINDING-AWARE" {
                       value 1;
                   }
                   enum "BINDING-INDEPENDENT" {
                       value 2;
                   }
               }
               description
                   "Data format:-binding-aware or binding-independent";
           }
           leaf transaction-type {
               mandatory true;
               type enumeration {
                   enum "SIMPLE-TX" {
                       value 1;
                   }
                   enum "TX-CHAINING" {
                       value 2;
                   }
               }
               description
                   "Data format:-binding-aware or binding-independent";
           }
           leaf outerElements {
               type uint32;
               default 100000;
               description
                 "Number of elements in the OuterList";
             }
           leaf innerElements {
               type uint32;
               default 1;
               description
                 "Number of elements in the InnerList";
             }
           leaf putsPerTx {
               type uint32;
               default 1;
               description
                 "Number of write operations (PUT, MERGE, or DELETE) per transaction submit";
             }
       }
       output {
           leaf status {
               mandatory true;
               type enumeration {
                   enum "OK" {
                       value 1;
                   }
                   enum "FAILED" {
                       value 2;
                   }
                   enum "TEST-IN-PROGRESS" {
                       value 3;
                   }
               }
               description
                   "Indicates whether the test finished successfuly";
              }
           leaf listBuildTime {
               type uint32;
               default 1;
               description
                 "The time it took to build the list of lists";
             }
           leaf execTime {
               type uint32;
               default 1;
               description
                 "The time it took to execute all transactions";
             }
           leaf txOk {
               type uint32;
               default 1;
               description
                 "The number of successful transactions";
             }
           leaf txError {
               type uint32;
               default 1;
               description
                 "The number of failed transactions";
             }
       }
   }

Binding Aware Data Format: Translating Yang Models Into Java POJOs and APIs

First, we'll look at how the data definitions in the test-exec container get translated into Java API and POJO definitions, which basically implement the Binding Aware data format for your yang model. The POJO definitions, along with their convenience functions, are generated by the Yang Tools at your application's compile time. You use these POJOs to pass data between your application and the MD-SAL.]. Note that POJOs are generated for all data definitions in your yang model, but in this section we will only look at those generated for the test-exec container.

Our model defines a list of lists that is stored in the configuration data store. There are basically two object definitions in the test-exec portion of our model: list outer-list {...} and list inner-list {...}. Yang Tools generated a package for each of them:

  • outer-list: ./dsbenchmark-api/src/main/yang-gen-sal/org/opendaylight/yang/gen/v1/urn/opendaylight/params/xml/ns/yang/dsbenchmark/rev150105/test/exec, and
  • inner-list: dsbenchmark-api/src/main/yang-gen-sal/org/opendaylight/yang/gen/v1/urn/opendaylight/params/xml/ns/yang/dsbenchmark/rev150105/test/exec/outer/list/

respectively. We use the POJO and API definitions in these packages in our code when building our list of lists. The code is in dsbenchmark-impl/src/main/java/org/opendaylight/dsbenchmark/DatastoreBaAbstractWrite.java.

First, for each element of the outer list, we first build the inner list:

   private List<InnerList> buildInnerList( int index, int elements ) {
       List<InnerList> innerList = new ArrayList<InnerList>( elements );

       for( int i = 0; i < elements; i++ ) {
           innerList.add(new InnerListBuilder()
                               .setKey( new InnerListKey( i ) )
                               .setName(i)
                               .setValue( "Item-"
                                          + String.valueOf( index )
                                          + "-"
                                          + String.valueOf( i ) )
                               .build());
       }
       return innerList;
   }

The objects in bold are defined in the inner list package. InnerList is the API that MD-SAL will use to access data from our Inner list objects. This is basically the shared API between MD-SAL and our application. MD-SAL "learns" this API when our model gets loaded into the controller. For each element in our inner list we use the InnerListBuilder convenience class to build a Data Transfer Object which is an implementation of the InnerList API. We also use the InnerListKey convenience class to build the list key for each inner list item.

Second, we build the outer list itself:

   private List<OuterList> buildOuterList( StartTestInput input ) {

      ... 
 
      List<OuterList> outerList = new ArrayList<OuterList>(input.getOuterElements().intValue());
       for( int j = 0; j < input.getOuterElements().intValue(); j++ ) {
           outerList.add(new OuterListBuilder()
                               .setId( j )
                               .setInnerList( buildInnerList( j, input.getInnerElements().intValue()) )
                               .setKey(new OuterListKey( j ))
                               .build() );
       }
 
      ...
 
      return outerList;
   }

The objects in bold are defined in the Outer List package. The purpose and operation of APIs and classes defined in the package are basically same as those defined for the inner list.

Ok, so we now built the entire list of lists - but how do we put it into the MD-SAL's data store? To find out, read Using the Binding-Aware Data Broker.

Binding Independent Data Format: The Data Object Model (DOM) for Yang

We do not use generated POJOs and APIs, but we still use QNAMEs generated for containers and data defined in the yang model. (a QNAME identifies the path to an object in the data store's data tree). Basically, we manipulate nodes in the data tree structures

We first create the node identifiers ("paths") that identify the inner list and outer list elements in the data store. The QNAME (path identifier) for the InnerList is defined in dsbenchmark-api/src/main/yang-gen-sal/org/opendaylight/yang/gen/v1/urn/opendaylight/params/xml/ns/yang/dsbenchmark/rev150105/ (a generated directory) as follows:

   public static final QName QNAME = org.opendaylight.yangtools.yang.common.QName.create("urn:opendaylight:params:xml:ns:yang:dsbenchmark","2015-01-05","inner-list");

In DatastoreDomAbstractWrite.java we add the QNAMEs for nodes that represent the InnerList element's leafs ("name" and "value") as follows:

   private static final org.opendaylight.yangtools.yang.common.QName IL_NAME = QName.create(InnerList.QNAME, "name");
   private static final org.opendaylight.yangtools.yang.common.QName IL_VALUE = QName.create(InnerList.QNAME, "value");

The QNAME (path identifier) for the OuterList List (i.e. the node in the data store tree graph that anchors the outer list in dsbenchmark's namepsace) is defined in dsbenchmark-api/src/main/yang-gen-sal/org/opendaylight/yang/gen/v1/urn/opendaylight/params/xml/ns/yang/dsbenchmark/rev150105/test/exec/' (a generated directory) as follows:

   public static final QName QNAME = org.opendaylight.yangtools.yang.common.QName.create("urn:opendaylight:params:xml:ns:yang:dsbenchmark","2015-01-05","outer-list");

Also in DatastoreDomAbstractWrite.java, we add the QNAME for the node that represents the InnerList in an OuterList element:

   private static final org.opendaylight.yangtools.yang.common.QName OL_ID = QName.create(InnerList.QNAME, "id");

Now we create the inner list:

   private MapNode buildInnerList( int index, int elements ) {
       CollectionNodeBuilder<MapEntryNode, MapNode> innerList = ImmutableNodes.mapNodeBuilder(InnerList.QNAME);
       for( int i = 0; i < elements; i++ ) {
           innerList.addChild(ImmutableNodes.mapEntryBuilder()
                               .withNodeIdentifier(new NodeIdentifierWithPredicates(InnerList.QNAME, IL_NAME, i))
                               .withChild(ImmutableNodes.leafNode(IL_NAME, i))
                               .withChild(ImmutableNodes.leafNode(IL_VALUE, "Item-"
                                                                            + String.valueOf(index)
                                                                            + "-"
                                                                            + String.valueOf(i)))
                               .build());
       }
       return innerList.build();
   }

Basically, we need to create a subtree that will hold all InnerList elements. The subtree is anchored in a single node returned by the buildInnerList function, which contains a node for each InnerList element, which in turn contains a node for each of its leafs (name and value).

Now we create the outer list:

   private List<MapEntryNode> buildOuterList( StartTestInput input ) {
       long startTime = System.nanoTime();
       List<MapEntryNode> outerList = new ArrayList<MapEntryNode>(input.getOuterElements().intValue());
       for( int j = 0; j < input.getOuterElements().intValue(); j++ ) {
           outerList.add(ImmutableNodes.mapEntryBuilder()
                               .withNodeIdentifier(new NodeIdentifierWithPredicates(OuterList.QNAME, OL_ID, j))
                               .withChild(ImmutableNodes.leafNode(OL_ID, j))
                               .withChild(buildInnerList(j, input.getInnerElements().intValue())) 
                               .build());
       }
       long endTime = System.nanoTime();
       listBuildTime = (endTime - startTime )/1000000;

       return outerList;
   }

Creation of the OuterList is similar to the creation of InnerList.

Implementing the Application's REST API

In DsbenchmarkProvider.java. The file generated by the MD-SAL application archetype was modified to implement the DsbenchmarkService interface, which in turn was generated by Yang Tools from the Service Model's RPC definitions. You can the DsbenchmarkService.java file in dsbenchmark-api/src/main/yang-gen-sal/org/opendaylight/yang/gen/v1/urn/opendaylight/params/xml/ns/yang/dsbenchmark/rev150105/ after running mvn generate-sources from dsbenchmark's root directory.

The modification and the service implementation are as shown below.

...
public class DsbenchmarkProvider implements BindingAwareProvider, DsbenchmarkService, AutoCloseable {
    ...
   @Override
   public Future<RpcResult<StartTestOutput>> startTest(StartTestInput input) {
       LOG.info("Starting the data store benchmark test, input: {}", input);
       // Check if there is a test in progress
       if ( execStatus.compareAndSet(ExecStatus.Idle, ExecStatus.Executing) == false ) {
           LOG.info("Test in progress");
           return RpcResultBuilder.success(new StartTestOutputBuilder()
                                                   .setStatus(StartTestOutput.Status.TESTINPROGRESS)
                                                   .build()).buildFuture();
       }
       // Cleanup data that may be left over from a previous test run
       cleanupTestStore();
        
       // Get the appropriate writer based on operation type and data format
       DatastoreWrite dsWriter = getDatastoreWrite(input);

       // Run the tests
       long startTime, endTime;
       try {
           startTime = System.nanoTime();
           dsWriter.writeList();
           endTime = System.nanoTime();
            this.testsCompleted++;
        } catch ( Exception e ) {
           LOG.error( "Test error: {}", e.toString());
           execStatus.set( ExecStatus.Idle );
           return RpcResultBuilder.success(new StartTestOutputBuilder()
                                                   .setStatus(StartTestOutput.Status.FAILED)
                                                   .build()).buildFuture();
       }
        LOG.info("Test finished");
       setTestOperData( ExecStatus.Idle, testsCompleted);
       execStatus.set(ExecStatus.Idle);

       StartTestOutput output = new StartTestOutputBuilder()
                                       .setStatus(StartTestOutput.Status.OK)
                                       .setListBuildTime(dsWriter.getListBuildTime())
                                       .setExecTime((endTime - startTime) / 1000000)
                                       .setTxOk((long)dsWriter.getTxOk())
                                       .setTxError((long)dsWriter.getTxError())
                                       .build();
       return RpcResultBuilder.success(output).buildFuture();
   }

   private void cleanupTestStore() {
       TestExec data = new TestExecBuilder()
                               .setOuterList(Collections.<OuterList>emptyList())
                               .build();
       WriteTransaction tx = dataBroker.newWriteOnlyTransaction();
       tx.put(LogicalDatastoreType.CONFIGURATION, TEST_EXEC_IID, data);
       try {
           tx.submit().checkedGet();
           LOG.info("DataStore test data cleaned up");
       } catch (TransactionCommitFailedException e) {
           LOG.info("Failed to cleanup DataStore test data");
           throw new IllegalStateException(e);
       }
   }
    ...
}

We have to tell MD-SAL about our test-exec API implementation so that it can route requests to it. This is done at application initialization in dsbenchmark-impl/src/main/java/org/opendaylight/yang/gen/v1/urn/opendaylight/params/xml/ns/yang/dsbenchmark/impl/rev141210/DsbenchmarkImplModule.java, in the createInstance() function:

   @Override
   public java.lang.AutoCloseable createInstance() {
       DsbenchmarkProvider provider = new DsbenchmarkProvider(getDomDataBrokerDependency());
       getBrokerDependency().registerProvider(provider);
       return provider;
   }

Simple Transactions

The data store APIs are designed around Guava Futures. You can interact with the data store in a synchronous or asynchronous manner. In the synchronous manner, your application is blocked when you submit a transaction until the transaction is completed. In the asynchronous manner, you specify a callback function which is invoked when the transaction finishes; your application can submit subsequent transactions without waiting for them to finish, but you have to use transaction chains and the Ping-Pong Broker to do that safely. In both cases you can use the binding-aware or the DOM format.

The Binding-Aware Data Broker

Initialization

For simple synchronous transactions that use the Binding Aware format, our app will talk to the Binding Aware OSGI Broker that comes pre-configured from the MD-SAL Startup Project Archetype.

Let's recap the initialization - first, the broker definition in the app's configuration yang model (dsbenchmark-impl/src/main/yang/dsbenchmark-impl.yang) is:

   ...
   augment "/config:modules/config:module/config:configuration" {
       case dsbenchmark-impl {
           when "/config:modules/config:module/config:type = 'dsbenchmark-impl'";
           container broker {
               uses config:service-ref {
                   refine type {
                       mandatory true;
                       config:required-identity md-sal-binding:binding-broker-osgi-registry;
                   }
               }
           }
       }
   }
   ....

which has the corresponding stanza in the default initial configuration (dsbenchmark-impl/src/main/config/default-config.xml):

<snapshot>
  ...
  <configuration>
     <modules xmlns="urn:opendaylight:params:xml:ns:yang:controller:config">
       ...
       <module>
         <type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:dsbenchmark:impl">prefix:dsbenchmark-impl</type>
         <name>dsbenchmark-default</name>
         <broker>
           <type xmlns:binding="urn:opendaylight:params:xml:ns:yang:controller:md:sal:binding">binding:binding-broker-osgi-registry</type>
           <name>binding-osgi-broker</name>
         </broker>
         ...
       </module>
     </modules>
   </data>
 </configuration>
</snapshot>

Finally, the default broker is retrieved at initialization from the app's context:

   public void onSessionInitiated(ProviderContext session) {
       this.dataBroker = session.getSALService(DataBroker.class);
       this.dstReg = session.addRpcImplementation( DsbenchmarkService.class, this );
       setTestOperData(this.execStatus.get(), testsCompleted);

       LOG.info("DsbenchmarkProvider Session Initiated");
   }

Using the Data Broker

For data in Binding Aware format, our app will talk to the Binding Aware broker. To write our OuterList element by element into the data store, we use the following code:

   @Override
   public void writeList() {
       WriteTransaction tx = dataBroker.newWriteOnlyTransaction();
       ...
       for (OuterList element : this.list) {
           InstanceIdentifier<OuterList> iid = InstanceIdentifier.create(TestExec.class)
                                                   .child(OuterList.class, element.getKey());
           tx.put(LogicalDatastoreType.CONFIGURATION, iid, element);
           ...
           try {
               tx.submit().checkedGet();
               ...
           } catch (TransactionCommitFailedException e) {
               LOG.error("Transaction failed: {}", e.toString());
               ...
           }
       }
   }

The example code above uses the "PUT" operation (the code in the repo is bit more complex: it uses either "PUT" or "MERGE", depneding on the parameter issued by the user). "PUT" replaces existing data in the data store with the subtree specified in th transaction, , "MERGE" will merge the existing subtree in the data store with the tree specified in the transaction. We use the synchronous future - our code waits for a transaction to finish before a next transaction is started. For each element in the outer list, we create an instance identifier for the element, "put" (or "merge") the element on the transaction and submit the transaction to the data store (note that we can do multiple PUTs/MERGEs per submit to improve performance - see the code in the repo and the performance charts later in this section).

The DOM Broker

Prerequisites

Maven Build Files

The pom.xml only contains dependencies for the binding-aware Data Broker. to use the DOM Broker, we need to add its dependencies to dsbenchmark-impl/pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

 <parent>
   <groupId>org.opendaylight.controller</groupId>
   <artifactId>config-parent</artifactId>
   <version>0.3.0-SNAPSHOT</version>
   <relativePath/>
 </parent>
 <modelVersion>4.0.0</modelVersion>
 <groupId>org.opendaylight.dsbenchmark</groupId>
 <artifactId>dsbenchmark-impl</artifactId>
 <version>1.0.0-SNAPSHOT</version>
 <packaging>bundle</packaging>
 <dependencies>
   <dependency>
     <groupId>${groupId}</groupId>
     <artifactId>dsbenchmark-api</artifactId>
     <version>${project.version}</version>
   </dependency>
   <dependency>
     <groupId>org.opendaylight.controller</groupId>
     <artifactId>sal-core-api</artifactId>
   </dependency>
   <dependency>
     <groupId>org.opendaylight.yangtools</groupId>
     <artifactId>yang-data-impl</artifactId>
   </dependency>
 </dependencies>
</project>
The Configuration Model and Default Config File

We need to add wiring for the DOM Data Broker - this is done by extending the configuration model and providing the template for the initial configuration.

Add DOM Broker configuration to the basic configuration yang model generated by the archetype:

module dsbenchmark-impl {
   yang-version 1;
   namespace "urn:opendaylight:params:xml:ns:yang:dsbenchmark:impl";
   prefix "dsbenchmark-impl";

   import config { prefix config; revision-date 2013-04-05; }
   import opendaylight-md-sal-binding { prefix md-sal-binding; revision-date 2013-10-28;}
   import opendaylight-md-sal-dom { prefix md-sal-dom; revision-date 2013-10-28;}

   description
       "Service definition for dsbenchmark project";

   revision "2014-12-10" {
       description
           "Initial revision";
   }
   identity dsbenchmark-impl {
       base config:module-type;
       config:java-name-prefix DsbenchmarkImpl;
   }
    augment "/config:modules/config:module/config:configuration" {
       case dsbenchmark-impl {
           when "/config:modules/config:module/config:type = 'dsbenchmark-impl'";
           container broker {
               uses config:service-ref {
                   refine type {
                       mandatory true;
                       config:required-identity md-sal-binding:binding-broker-osgi-registry;
                   }
               }
           }
           container dom-data-broker {
               uses config:service-ref {
                   refine type {
                       mandatory true;
                       config:required-identity md-sal-dom:dom-async-data-broker;
                   }
               }
           }
       }
   }
}

Add DOM Broker initial configuration to the basic default configuration file generated by the archetype:

<snapshot>
 <required-capabilities>
 </required-capabilities>
 <configuration>
   
     <modules xmlns="urn:opendaylight:params:xml:ns:yang:controller:config">
       <module>
         <type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:dsbenchmark:impl">prefix:dsbenchmark-impl</type>
         <name>dsbenchmark-default</name>
         <broker>
           <type xmlns:binding="urn:opendaylight:params:xml:ns:yang:controller:md:sal:binding">binding:binding-broker-osgi-registry</type>
           <name>binding-osgi-broker</name>
         </broker>
         <dom-data-broker>
           <type xmlns:dom="urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom">dom:dom-async-data-broker</type>
           <name>inmemory-data-broker</name>
         </dom-data-broker>
       </module>
     </modules>
   
 </configuration>
</snapshot>
Runtime Initialization

To be able to create DOM Transactions, we need a reference to the DOM Data Broker. Unfortunately, it is not available from the session in onSessionInitialized, so we have to pass it into our DsbenchmarkProvider at its creation time. We modify the createInstance() method in DsbenchmarkImplModule.java as follows:

public java.lang.AutoCloseable createInstance() {
    DsbenchmarkProvider provider = new DsbenchmarkProvider(getDomDataBrokerDependency());
    getBrokerDependency().registerProvider(provider);
    return provider;
}
Using the DOM Broker

We interact with the DOM Broker as follows:

...
DOMDataWriteTransaction tx = domDataBroker.newWriteOnlyTransaction();
long putCnt = 0;

for (MapEntryNode element : this.list) {
    YangInstanceIdentifier yid = YangInstanceIdentifier.builder().node(TestExec.QNAME)
                                               .node(OuterList.QNAME)
                                               .nodeWithKey(OuterList.QNAME, element.getIdentifier().getKeyValues())
                                               .build();            
    tx.put(LogicalDatastoreType.CONFIGURATION, yid, element);
    try {
        tx.submit().checkedGet();
    } catch (TransactionCommitFailedException e) {
        LOG.error("Transaction failed: {}", e.toString());
    }
}
...

This is a mirror image to the Binding-Aware Broker code; but rather than using WriteTransactions we use DOMDataWriteTransactions, and instead of InstanceIdentifiers<> we use the YangInstanceIdentifiers. Of course, we use the data tree we built earlier, rather than binding-aware DTOs.

Transaction Chains & the Ping-Pong Broker

When you want your data store transaction to finish asynchronously (and let your application's thread do other work in the meaintime), you must use transaction chains and the Pingpong Broker. Transaction chains will ensure that when you issue a sequence of transactions and transactions in the sequence finish out of order - it's quite common for transactions to take vastly different times to finish, see the benchmarks later in this tutorial. The pingpong buffer will provide the optimal performance and back-pressure to your app when the data store transaction queue becomes full.

Prerequisites

The Configuration Model and Default Config File

To make the binding-aware pingpong broker available to our app, we need to add the wiring configuration for the binding-aware async data broker to our app's config model (dsbenchmark/dsbenchmark-impl/src/main/config/default-config.xml): We extend it with the following stanza:

container binding-data-broker {
    uses config:service-ref {
        refine type {
            mandatory true;
            config:required-identity md-sal-binding:binding-async-data-broker;
        }
    }
}

The configuration model already supports the DOM pingpong broker - we just need to change the default config file. The default config file with the DOM and Binding Pingpong Broker configurations is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<snapshot>
 <required-capabilities>
   <capability>urn:opendaylight:params:xml:ns:yang:dsbenchmark:impl?module=dsbenchmark-impl&revision=2014-12-10</capability>
   <capability>urn:opendaylight:params:xml:ns:yang:controller:md:sal:binding?module=opendaylight-md-sal-binding&revision=2013-10-28</capability>
 </required-capabilities>
 <configuration>
   
     <modules xmlns="urn:opendaylight:params:xml:ns:yang:controller:config">
       <module>
         <type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:dsbenchmark:impl">prefix:dsbenchmark-impl</type>
         <name>dsbenchmark-default</name>
         <broker>
           <type xmlns:binding="urn:opendaylight:params:xml:ns:yang:controller:md:sal:binding">binding:binding-broker-osgi-registry</type>
           <name>binding-osgi-broker</name>
         </broker>
         <dom-data-broker>
           <type xmlns:dom="urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom">dom:dom-async-data-broker</type>
           <name>pingpong-broker</name>
         </dom-data-broker>
         <binding-data-broker>
           <type xmlns:binding="urn:opendaylight:params:xml:ns:yang:controller:md:sal:binding">binding:binding-async-data-broker</type>
           <name>pingpong-binding-data-broker</name>
         </binding-data-broker>
       </module>
     </modules>
   
 </configuration>
</snapshot>

Runtime Initialization

To be able to create transactions in a Transaction Chain, we need a reference to the Binding Data Broker. Just like the DOM Data Broker, it is not available from the session in onSessionInitialized, so we have to pass it into our DsbenchmarkProvider at its creation time. We modify the createInstance() method in dsbenchmark/dsbenchmark-impl/src/main/java/org/opendaylight/yang/gen/v1/urn/opendaylight/params/xml/ns/yang/dsbenchmark/impl/rev141210/DsbenchmarkImplModule.java as follows:

   public java.lang.AutoCloseable createInstance() {
       DsbenchmarkProvider provider = new DsbenchmarkProvider(getDomDataBrokerDependency(),
                                                              getBindingDataBrokerDependency());
       getBrokerDependency().registerProvider(provider);
       return provider;
   }

We also need to store the reference to Binding Data Broker DsbenchmarkProvider and change its constructor as follows:

   public DsbenchmarkProvider(DOMDataBroker domDataBroker, DataBroker bindingDataBroker) {
       ...
       this.domDataBroker = domDataBroker;
       this.bindingDataBroker = bindingDataBroker;
   }

Implementing the TransactionChainListener Interface

A component in your code needs to implement the TransactionChainListener interface to receive notifications related to the transaction chains chains you create. You register with the Data Broker when you create your transaction chain. For example, the tutorial code for Binding-aware transaction chain uses the TxchainBaWrite classes to do this:

public class TxchainBaWrite extends DatastoreAbstractWriter implements TransactionChainListener {
   ...
   public void writeList() {
       BindingTransactionChain chain = bindingDataBroker.createTransactionChain(this);
   }
   @Override
   public void onTransactionChainFailed(TransactionChain<?, ?> chain,
           AsyncTransaction<?, ?> transaction, Throwable cause) {
       LOG.error("Broken chain {} in TxchainDomWrite, transaction {}, cause {},
               chain, transaction.getIdentifier(), cause);
   }
   @Override
   public void onTransactionChainSuccessful(TransactionChain<?, ?> chain) {
       LOG.info("Chain {} closed successfully", chain);
   }

}

Using Binding-Aware Transaction Chains

Once you've done the wiring and initialization, using transaction chains is simple: rather than getting your transactions from a Data Broker, you get them from the chain. That's it. The code below illustrates the point:

   @Override
   public void writeList() {
       BindingTransactionChain chain = bindingDataBroker.createTransactionChain(this);
       WriteTransaction tx = chain.newWriteOnlyTransaction();

       for (OuterList element : this.list) {
           InstanceIdentifier<OuterList> iid = InstanceIdentifier.create(TestExec.class)
                                                   .child(OuterList.class, element.getKey());
           tx.put(LogicalDatastoreType.CONFIGURATION, iid, element);
           Futures.addCallback(tx.submit(), new FutureCallback<Void>() {
               @Override
               public void onSuccess(final Void result) {
                   ...
               }
               @Override
               public void onFailure(final Throwable t) {
                   ...
               }
           });
       }
       ....
   }

In most cases, you don't ever need to close your transaction chain - your app will need to write to the data store for the app's entire lifetime. But, if writing into the data store is a one-off, you need to close the transaction chain properly. The tutorial code shows how to do it:

   @Override
   public void writeList() {

       ...

       // *** Clean up and close the transaction chain ***
       // Submit the outstanding transaction even if it's empty and wait for it to finish
       // We need to empty the transaction chain before closing it
       try {
           tx.submit().checkedGet();
       } catch (TransactionCommitFailedException e) {
           ...
       }
       try {
           chain.close();
       }
       catch (IllegalStateException e){
           LOG.error("Transaction close failed,", e);
       }
       ....
   }

Using DOM Transaction Chains

Using DOM transaction chains is very similar to using Binding-Aware transaction chains: instead of creating a BindingTransactionChain you create a DOMTransactionChain, which you will use to get DOMDataWriteTransactions. The code below illustrates the point:

   public void writeList() {
       DOMTransactionChain chain = domDataBroker.createTransactionChain(this);
       DOMDataWriteTransaction tx = chain.newWriteOnlyTransaction();
       YangInstanceIdentifier pid = YangInstanceIdentifier.builder().node(TestExec.QNAME).node(OuterList.QNAME).build();

       for (MapEntryNode element : this.list) {
           YangInstanceIdentifier yid = pid.node(new NodeIdentifierWithPredicates(OuterList.QNAME, element.getIdentifier().getKeyValues()));

           tx.put(LogicalDatastoreType.CONFIGURATION, yid, element);
           Futures.addCallback(tx.submit(), new FutureCallback<Void>() {
               @Override
               public void onSuccess(final Void result) {
                   ...
               }
               @Override
               public void onFailure(final Throwable t) {
                   ...
                }
           });
       }
   }

Benchmark Results

Analyzing Data Layout, Data Format and Operation Type

We used the dsbenchmark.py script to collect performance data. The benchmark was run on a Mid-2014 MacBook Pro with 16 Gig of RAM and 2.2 Gig i7 processor. The table shows execution times for writing (or deleting) a list of lists with 100k inner elements for different list-of-lists layouts. For example, the first column shows a list with 100k outer elements and each outer element has an inner list with a single inner-list element. The second column shows a list with 10k outer elements and each outer element has an inner list with 10 inner-list elements.

Outer/Inner list -> 100000/1 10000/10 1000/100 100/1000 10/10000 1/100000
BA PUT 3840.8 436 178.2 157.3 156 154.6
BA MERGE 4475 706.8 430.6 435.4 580.2 729.7
BA DELETE 3336.9 395.9 154.7 151.2 154.8 160.4
DOM PUT 3358.7 310.7 74.3 68.7 71.2 82.6
DOM MERGE 4802.5 636.9 380.7 423.4 536.6 666.3
DOM DELETE 3345.9 418.2 134.3 129.7 143.9 141

The table is shown in a graphical format in the following figure:

DS Benchmark1.png

In terms of transactions/second, data in the above table yields the following graphs:

DS Benchmark2.png

Data Layout: Number of Outer vs. Inner Elements

The effects of data layout are shown in the graphs in the previous section. There is one more useful view that we give here - data throughput measured as the number of inner elements written into the data store per second. This is shown in the following graph:

DS Benchmark3.png

Data Format: Binding-Aware vs. DOM

The relative data store performance of using the DOM format vs. the Binding Aware format for the PUT and MERGE operations is shown in the following graph:

DS Benchmark4.png

As can be seen from the graph, for our data set, using the DOM format can result in performance up to 2.5 higher for PUT operations, while the performance for MERGE operations is roughly the same.

Write Operation Type: PUT vs. MERGE

The relative data store performance of using the PUT operation vs. the MERGE operation for the DOM and BINDING-AWARE data formats is shown in the following graph:

DS Benchmark5.png

As can be seen from the graph, for our data set, using the MERGE operation is quite expensive, compared to PUT and becomes more expensive as the size of the inner list grows. MERGE has more work to do if the trees it is merging are more complex..

Analyzing Batching of Data Store Operations {PUT, MERGE, DELETE}

We can issue multiple write or delete operations per single transaction. In this section we use a list of lists with 100k outer elements and each outer element has an inner list with a single inner element. We analyze performance for different levels of batching - from doing one write (PUT or MERGE) per transaction to doing all writes in a single transaction. the execution times for these benchmark tests are shown in the following figure:

DS Benchmark6.png

The graph shows that batching can significantly improve the time it takes to write the list into the data store. The transactions per second number is not htat interesting, since the transaction vary in complexity. Another view of the data - the throughput measured as the number of inner elements written into the data store per second- is given in the following graph:

DS Benchmark7.png

The graph shows that the application's throughput can be improved by up to an order of magnitude by carefully batching write operations into transactions.