Jump to: navigation, search

OpenDaylight Controller:MD-SAL:Migration:Data Broker

In Helium timeframe MD-SAL is going to change Data Broker APIs based on user feedback, and new Data Broker APIs we're designed with community.

All Data APIs from package org.opendaylight.controller.sal.binding.api.data and org.opendaylight.controller.sal.core.api.data are going to be deprecated and replaced with org.opendaylight.controller.md.sal.binding.api and org.opendaylight.controller.md.sal.dom.api

Implementation of MD-SAL already uses new DOM Data Broker and DOM APIs and provides support for legacy APIs for more then two months.

Important changes

  • Any data operations must be done using Transactions
  • Transaction Chaining
  • Read are not blocking, but asynchronous
  • Clarified difference between put (add/replace) and merge
  • Clarified APIs
  • Granular registration options for data change listeners.

Migration Guide

Please consult Javadocs for org.opendaylight.controller.md.sal.binding.api.DataBroker which contains detailed description about new APIs alongside with examples.

Migrating DataChangeListener

If you have DataChangeListener(s) in your code this is what you should do to migrate them to the new Data Broker

Substitute

    @Override
    public void onDataChanged(
            DataChangeEvent<InstanceIdentifier<?>, DataObject> change) {

by

    @Override
    public void onDataChanged(
            final AsyncDataChangeEvent<InstanceIdentifier<?>, DataObject> change ) {

DataChangeListener Registration

Before


        final ListenerRegistration<DataChangeListener> sfEntryDataChangeListenerRegistration =
                dataBrokerService.registerDataChangeListener( OpendaylightSfc.sfEntryIID, 
                        sfcProviderSfEntryDataListener);

After


        final ListenerRegistration<DataChangeListener> sfEntryDataChangeListenerRegistration =
                dataBrokerService.registerDataChangeListener( LogicalDatastoreType.CONFIGURATION,
                        OpendaylightSfc.sfEntryIID, sfcProviderSfEntryDataListener, DataBroker.DataChangeScope.SUBTREE);


Imports

Fixing imports can be frustrating because methods can have similar names, or it is not clear which ones to add or remove.

As far as imports go, you should certainly remove all instances of:

import org.opendaylight.controller.sal.binding.api.data.DataChangeListener;
import org.opendaylight.controller.sal.binding.api.data.DataProviderService;

and add the following whenever appropriate.

import org.opendaylight.controller.md.sal.common.api.data.AsyncDataChangeEvent;
import org.opendaylight.controller.md.sal.binding.api.DataBroker;
import org.opendaylight.controller.md.sal.binding.api.DataChangeListener;

If you are going to perform data store transactions, also add right off the bat:


import com.google.common.base.Optional;
import org.opendaylight.controller.md.sal.binding.api.ReadOnlyTransaction;
import org.opendaylight.controller.md.sal.binding.api.ReadWriteTransaction;
import org.opendaylight.controller.md.sal.binding.api.WriteTransaction;
import org.opendaylight.controller.md.sal.common.api.data.LogicalDatastoreType;

Do not worry about unused imports, most IDEs can optimize them so the code looks sharp.

Migrating Read Transactions

Now let me show how to migrate an actual read transaction

Before

        InstanceIdentifier<ServiceFunctionForwarder> sffIID;
        sffIID = InstanceIdentifier.builder(ServiceFunctionForwarders.class)
                .child(ServiceFunctionForwarder.class, serviceFunctionForwarderKey)
                .build();
        DataObject dataObject = odlSfc.dataProvider.readConfigurationData(sffIID);
        if (dataObject instanceof ServiceFunctionForwarder) {
            ServiceFunctionForwarder serviceFunctionForwarder = (ServiceFunctionForwarder) dataObject;
            return serviceFunctionForwarder;
        } else {
            return null;
        }

After


        sffIID = InstanceIdentifier.builder(ServiceFunctionForwarders.class)
                .child(ServiceFunctionForwarder.class, serviceFunctionForwarderKey)
                .build();

        ReadOnlyTransaction readTx = odlSfc.dataProvider.newReadOnlyTransaction();
        Optional<ServiceFunctionForwarder> serviceFunctionForwarderObject = null;
        try {
            serviceFunctionForwarderObject = readTx.read(LogicalDatastoreType.CONFIGURATION, sffIID).get();
        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }

        if (serviceFunctionForwarderObject.get() instanceof ServiceFunctionForwarder) {
            LOG.debug("\n########## Stop: {}", Thread.currentThread().getStackTrace()[1]);
            return serviceFunctionForwarderObject.get();
        } else {
            LOG.debug("\n########## Stop: {}", Thread.currentThread().getStackTrace()[1]);
            return null;
        }

There are some very important things to notice:

  • You have to supply an actual class to Optional<> in order for the code to behave. You can not use the generic Optional<DataObject> since the return of your read is not a DataObject
  • The well-known construct if (dataObject instanceof ServiceFunctionForwarder) does not work anymore. If you want to test or use the results of the read you actually have to perform an extra step: serviceFunctionForwarderObject.get()
  • You have to use a try-catch, which means some code reshuffling is necessary in order to make sure things are processed within the try or catch blocks

Migrating Write (merge) Transactions

If you have legacy code that saved data in the datastore and relied implicitly or explicitly that:

  1. The API took care of creating all parents. In plain English this means that if you create a leaf node without creating the parents first, the API would take care of creating the parents (whether with or without data). A good analogy: the mkdir -p UNIX command.
  2. The API actually merged data instead of complete overwrite

Then you the following is needed

        WriteTransaction writeTx = odlSfc.dataProvider.newWriteOnlyTransaction();
        writeTx.merge(LogicalDatastoreType.CONFIGURATION,
                sffIID, serviceFunctionForwarderBuilder.build(), true);

Notice

  • The use of a merge transaction
  • The use of an extra parameter (true), so that parents are created.

Design Discussion: Blocking vs. Non-blocking Reads

Although non-blocking reads are discouraged, in some situations there is nothing you can do to recover if a transaction fails in a callback context. What this means is that a delayed failure or delayed success mean the same thing if the code depends on the answer of the read right there in order to proceed.

Reading Data

New DataBroker APIs requires all reads to happen inside transaction, and provides specific transaction type only for reading which could be allocated by invoking #newReadOnlyTransaction() on DataBroker.


Legacy API

Example bellow illustrates legacy read of data, which required null pointers check and was blocking (even if read was remote - e.g. Netconf Device).

        DataModificationTransaction readTx = legacyBroker.beginTransaction();
        DataObject data = readTx.readConfigurationData(PATH);

        if(data != null) {
            // data are present in data store.
            doSomething(data);
        } else {
            // data are not present in data store.
        }
New API
Blocking read

Note: Blocking reads are discouraged, try to use non-blocking reads as much as possible.

        ReadOnlyTransaction readTx = dataBroker.newReadOnlyTransaction();
        Optional<DataObject> data = readTx.read(LogicalDatastoreType.CONFIGURATION,PATH).get();
       
        if(data.isPresent()) {
           // data are present in data store.
            doSomething(data.get());
        } else {
            // data are not present in data store.
        }
Non-blocking read
        ReadOnlyTransaction readTx = dataBroker.newReadOnlyTransaction();
        ListenableFuture<Optional<DataObject>> dataFuture = readTx.read(LogicalDatastoreType.CONFIGURATION,PATH);
        Futures.addCallback(dataFuture, new FutureCallback<Optional<DataObject>>() {
            @Override
            public void onSuccess(final Optional<DataObject> result) {
                if(result.isPresent()) {
                 // data are present in data store.
                    doSomething(result.get());
                } else {
                    // data are not present in data store.
                }
            }
            @Override
            public void onFailure(final Throwable t) {
                // Error during read
            }
  • onSuccess() will be invoked once read is done and result is returned.


Modifying data

Behaviour of some methods in legacy API was not well-documented and naming of operations was sometimes missleading users who were used to REST operations.

New APIs have better documented APIs and better naming for operations.


Adding Data

Legacy API
        DataModificationTransaction writeTx = legacyBroker.beginTransaction();
        // write to configuration store
        writeTx.putConfigurationData(PATH, DATA);
        // write to operation store
        writeTx.putOperationalData(PATH, DATA);     
New API
        ReadWriteTransaction writeTx = dataBroker.newReadWriteTransaction();
        // replace in configuration store
        writeTx.put(LogicalDatastoreType.CONFIGURATION,PATH, DATA);
        // replace in operational store
        writeTx.put(LogicalDatastoreType.OPERATIONAL,PATH, DATA);       

Replacing Data

Put method in legacy APIs was used as "merge" method, which did not replace data, but merged them with previous state, which is inconsistent with expection of clients. replace was always done by remove/put combination.

Legacy API
        DataModificationTransaction writeTx = legacyBroker.beginTransaction();
        // replace in configuration store
        writeTx.removeConfigurationData(PATH);
        writeTx.putConfigurationData(PATH, DATA);
        // replace in operation store
        writeTx.removeOperationalData(PATH);
        writeTx.putOperationalData(PATH, DATA);
        
New API
        ReadWriteTransaction writeTx = dataBroker.newReadWriteTransaction();
        // replace in configuration store
        writeTx.put(LogicalDatastoreType.CONFIGURATION,PATH, DATA);
        // replace in operational store
        writeTx.put(LogicalDatastoreType.OPERATIONAL,PATH, DATA);


Merging Data

Legacy API
        DataModificationTransaction writeTx = legacyBroker.beginTransaction();
        // merge in configuration store
        writeTx.putConfigurationData(PATH, DATA);
        // merge in operational store
        writeTx.putOperationalData(PATH, DATA);
New API
        ReadWriteTransaction writeTx = dataBroker.newReadWriteTransaction();
        // merge in configuration store
        writeTx.merge(LogicalDatastoreType.CONFIGURATION,PATH, DATA);
        // merge in configuration store
        writeTx.merge(LogicalDatastoreType.OPERATIONAL,PATH, DATA);


Commiting Transaction

New Data Broker APIs provides better visibility into transaction eg. reason of failure and also ListenableFuture as result, which makes it easier to listen on asynchronous commit and its state.

API - Asynchronous Commit state
        ReadWriteTransaction writeTx = dataBroker.newReadWriteTransaction();

        CheckedFuture<Void,TransactionCommitFailedException> submitFuture = writeTx.submit();

        Futures.addCallback(submitFuture, new FutureCallback<Void>() {

            @Override
            public void onSuccess(final Void result) {
                // Commited successfully
            }

            @Override
            public void onFailure(final Throwable t) {
                // Transaction failed

                if(t instanceof OptimisticLockFailedException) {
                    // Failed because of concurrent transaction modifying same data
                } else {
                   // Some other type of TransactionCommitFailedException
                }
            }

        });
New API - Write/Commit/Retry Example

This example illustrates simple write/commit/retry example, when we are writing same data, but if commit failed because of concurrent modification, we will retry same transaction.


    private void readWriteRetry(final int tries) {
        ReadWriteTransaction writeTx = dataBroker.newReadWriteTransaction();
        
        Optional<DataObject> readed = writeTx.read(LogicalDatastoreType.OPERATIONAL,PATH).get();
        DataObject modified = modifyData(readed);
        writeTx.put(LogicalDatastoreType.OPERATIONAL, PATH, data);
        CheckedFuture<Void,TransactionCommitFailedException> future = writeTx.submit();

        Futures.addCallback(future, new FutureCallback<Void>() {

            @Override
            public void onSuccess(final Void result) {
                // Commited successfully
                // Nothing to do
            }

            @Override
            public void onFailure(final Throwable t) {
                // Transaction failed

                if(t instanceof OptimisticLockFailedException) {
                    if( (tries - 1) > 0 ) {
                        LOG.debug("Concurrent modification of data - trying again");
                        readWriteRetry(tries - 1);
                    }
                    else {
                        LOG.error("Concurrent modification of data - out of retries",e);
                    }
                }
            }
        });
    }