Jump to: navigation, search

Karaf 4 migration

This page explains the steps required to migrate a project to Karaf 4, taking the ttp project as an example.

Tracking

Step-by-step guide

Step 1: identify the projects depended upon for features

Look at the project's current features.xml file (or files, if there are several):

<features name="odl-ttp-${project.version}" xmlns="http://karaf.apache.org/xmlns/features/v1.2.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.2.0 http://karaf.apache.org/xmlns/features/v1.2.0">

    <repository>mvn:org.opendaylight.yangtools/features-yangtools/${yangtools.version}/xml/features</repository>
    <repository>mvn:org.opendaylight.mdsal.model/features-mdsal-model/${mdsal.model.version}/xml/features</repository>
    <repository>mvn:org.opendaylight.netconf/features-restconf/${restconf.version}/xml/features</repository>

    <feature name='odl-ttp-all' version='${project.version}' description='OpenDaylight :: ttp :: All'>
        <feature version='${project.version}'>odl-ttp-model</feature>
        <feature version='${project.version}'>odl-ttp-model-rest</feature>
    </feature>

    <feature name='odl-ttp-model' version='${project.version}' description='OpenDaylight :: ttp :: Model'>
        <feature version='${mdsal.model.version}'>odl-mdsal-models</feature>
        <bundle>mvn:org.opendaylight.ttp/ttp-model/${project.version}</bundle>
        <bundle>mvn:org.opendaylight.controller.model/model-inventory/${controller.version}</bundle>
    </feature>

    <feature name='odl-ttp-model-rest' version='${project.version}' description='OpenDaylight :: ttp :: Model'>
        <feature version='${project.version}'>odl-ttp-model</feature>
        <feature version='${restconf.version}'>odl-mdsal-apidocs</feature>
        <feature version='${restconf.version}'>odl-restconf</feature>
        <feature version='${controller.version}'>odl-mdsal-broker</feature>
    </feature>

</features>

This references three repositories: features-yangtools, features-mdsal-model and features-restconf. So we need to wait for the corresponding projects to switch; this will result in the availability of three new Karaf 4 repositories, features4-yangtools, features4-mdsal-model and features4-restconf.

You can determine other projects’ Karaf 4 status by looking for Karaf 4 artifacts in Nexus, or by checking their tracking bug in the bug tracker (start with #4219).

Step 2: prepare the new project layout

The desired project layout is as follows, within the features directory:

  • features becomes an aggregator project containing
    • features-ttp which defines the old features repository
    • features4-ttp which defines the new, Karaf 4, features repository
    • odl-ttp-all which defines the odl-ttp-all feature (for Karaf 4)
    • odl-ttp-model which defines the odl-ttp-model feature (for Karaf 4)
    • odl-ttp-model-rest which defines the odl-ttp-model-rest feature (for Karaf 4)

features-ttp will continue to provide its features. This results in features being defined twice with the same names, but within different feature repositories.

To prepare the new project layout, create a new features-ttp folder inside the features folder, move src there, and copy pom.xml. This goes from

Ttp-features-1.png

to

Ttp-features-2.png

Open the old pom.xml and turn it into an aggregator:

  • use odlparent-lite as parent POM
  • use packaging type "pom"
  • only list the modules we're going to end up with
  • change the artifactId (to "features-aggregator" typically)

Your first module is the old features module. Add the new modules too (then your IDE will help you create them). All this results in

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
   <modelVersion>4.0.0</modelVersion>
   <parent>
      <groupId>org.opendaylight.odlparent</groupId>
      <artifactId>odlparent-lite</artifactId>
      <version>1.8.0-SNAPSHOT</version>
      <relativePath/>
   </parent>
   <groupId>org.opendaylight.ttp</groupId>
   <artifactId>features-aggregator</artifactId>
   <version>0.4.0-SNAPSHOT</version>
   <packaging>pom</packaging>

   <modules>
      <module>features-ttp</module>
      <module>features4-ttp</module>
      <module>odl-ttp-all</module>
      <module>odl-ttp-model</module>
      <module>odl-ttp-model-rest</module>
   </modules>
</project>

(with the scm, url and distributionManagement elements as before, if your features POM had them).

Step 3: define the new features

Create the new odl- modules, using single-feature-parent as POM parent:

  • use packaging type "feature"
  • list all the features your feature depends upon, using type "xml" and classifier "features"
  • list all the bundles your feature contains
  • use your feature's description as the POM's name element

It will probably make your life easier if you define your features starting with the "bottom-most" one: that way you'll get help from your IDE as you build more complex features up. In ttp's case, we start with odl-ttp-model, then create odl-ttp-model-rest (which uses odl-ttp-model), and finally odl-ttp-all.

Since we're working with standard POMs here you can use dependency management, import artifact POMs etc. Note that if you depend directly on odlparent features, you'll need to use the odl4- variants here (odl4-guava-18 etc.).

The following sed script can help convert a features.xml file to a series of POM dependencies:

#!/usr/bin/sed -f

sX<feature version=['"]\([^'"]*\)['"]>\([^<]*\)</feature>X<dependency><groupId></groupId><artifactId>\2</artifactId><version>\1</version><type>xml</type><classifier>features</classifier></dependency>Xg
sX<bundle>mvn:\([^/]*\)/\([^/]*\)/Template:VERSION</bundle>X<dependency><groupId>\1</groupId><artifactId>\2</artifactId></dependency>Xg
sX<bundle>wrap:mvn:\([^/]*\)/\([^/]*\)/Template:VERSION</bundle>X<dependency><groupId>\1</groupId><artifactId>\2</artifactId></dependency>Xg
sX<bundle>mvn:\([^/]*\)/\([^/]*\)/\([^<]*\)</bundle>X<dependency><groupId>\1</groupId><artifactId>\2</artifactId><version>\3</version></dependency>Xg
sX<bundle>wrap:mvn:\([^/]*\)/\([^/]*\)/\([^<]*\)</bundle>X<dependency><groupId>\1</groupId><artifactId>\2</artifactId><version>\3</version></dependency>Xg

For odl-ttp-model the result is

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.opendaylight.odlparent</groupId>
        <artifactId>single-feature-parent</artifactId>
        <version>1.8.0-SNAPSHOT</version>
        <relativePath/>
    </parent>

    <groupId>org.opendaylight.ttp</groupId>
    <artifactId>odl-ttp-model</artifactId>
    <version>0.4.0-SNAPSHOT</version>
    <packaging>feature</packaging>

    <name>OpenDaylight :: ttp :: Model</name>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.opendaylight.mdsal.model</groupId>
                <artifactId>mdsal-model-artifacts</artifactId>
                <version>0.10.0-SNAPSHOT</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.opendaylight.controller</groupId>
                <artifactId>mdsal-artifacts</artifactId>
                <version>1.5.0-SNAPSHOT</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.opendaylight.mdsal.model</groupId>
            <artifactId>odl-mdsal-models</artifactId>
            <type>xml</type>
            <classifier>features</classifier>
        </dependency>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>ttp-model</artifactId>
            <version>${project.version}</version>
        </dependency>
        <dependency>
            <groupId>org.opendaylight.controller.model</groupId>
            <artifactId>model-inventory</artifactId>
        </dependency>
    </dependencies>
    
</project>

At this point you can build the module to check the generate feature.xml:

cd features/odl-ttp-model
mvn clean package

This will generate a feature.xml file in target/feature which you can inspect.

You can also run your feature through SingleFeatureTest:

mvn test -Psft

(The default profile doesn't run SingleFeatureTest; features are tested in their repositories.)

odl-ttp-model-rest and odl-ttp-all are left as an exercise for the reader.

Step 4: define the new features repository

Create the new features4-ttp module, using feature-repo-parent as POM parent:

  • use packaging type "feature"
  • list all the features as dependencies, with type "xml" and classifier "features"

The result for ttp is

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.opendaylight.odlparent</groupId>
        <artifactId>feature-repo-parent</artifactId>
        <version>1.8.0-SNAPSHOT</version>
    </parent>

    <groupId>org.opendaylight.ttp</groupId>
    <artifactId>features4-ttp</artifactId>
    <version>0.4.0-SNAPSHOT</version>
    <packaging>feature</packaging>

    <dependencies>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>odl-ttp-all</artifactId>
            <version>${project.version}</version>
            <type>xml</type>
            <classifier>features</classifier>
        </dependency>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>odl-ttp-model</artifactId>
            <version>${project.version}</version>
            <type>xml</type>
            <classifier>features</classifier>
        </dependency>
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>odl-ttp-model-rest</artifactId>
            <version>${project.version}</version>
            <type>xml</type>
            <classifier>features</classifier>
        </dependency>
    </dependencies>

</project>

Instead of repeating <version>${project.version}, you can add the odl-* features which are intended for external consumption to your project's dependencyManagement *-artifacts artifact, and then dependencyManagement <scope>import that into your new features repository (above).

At this point the tree looks like

Ttp-features-3.png

You can now build the full features module, which will build the old Karaf 3 features and the new Karaf 4 features, and run everything through SingleFeatureTest:

mvn clean package

Step 5: commit and review

That's it, your project's migration is complete; you should test your new features in a Karaf 4 distribution, and submit your patch for review, using the "karaf4" topic.

Step 6: clean up your features

(This step isn’t required for the Karaf 4 migration.)

As a follow-up to the Karaf 4 migration, it would probably be interesting to clean up the feature POM files: since Karaf Maven plugin adds the required bundles to your features to satisfy transitive dependencies, you no longer need to explicitly mention Maven artifacts which aren't "first-order" contents of your features. For example, in odl-ttp-model, I don't think we really want to include model-inventory in the feature in all cases; it's just a dependency of ttp-model, so we could remove it from the POM.

As a rule of thumb, your POM feature definitions should only include features you depend upon (which can come from other projects of course), and bundles you create (with your groupId); any other bundles should only be pulled in as transitive dependencies, unless they're actually part of your documented API (e.g. Guava in old4-guava-21). Also, if you end up with bundles from other ODL projects pulled in transitively, you should probably look for the appropriate feature and depend on that instead.

Step 7: test a project-specific Karaf 4 distribution

Many projects build a “local” Karaf distribution, using karaf-parent. This step switches that to Karaf 4, using karaf4-parent.

If your project already has a karaf-parent-based module, adapt it as follows:

  • replace karaf-parent with karaf4-parent in your POM’s parent;
  • use your Karaf 4 feature repository instead of the Karaf 3 feature repository in your dependencies;
  • make sure your local feature (the ``karaf.localFeature`` property) still makes sense.

As an example, #58788 switches TTP to the Karaf 4 parent (and cleans up the POM).

If your project doesn’t have a karaf-parent-based module yet, create a new module using karaf4-parent; you’ll need at least

 <?xml version="1.0" encoding="UTF-8"?>
 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
 
   <parent>
     <groupId>org.opendaylight.odlparent</groupId>
     <artifactId>karaf4-parent</artifactId>
     <version>1.8.0-Carbon</version>
     <relativePath/>
   </parent>
 
   <groupId>org.opendaylight.ttp</groupId>
   <artifactId>distribution-karaf</artifactId>
   <version>0.5.0-SNAPSHOT</version>
 
   <name>ODL :: ttp :: ${project.artifactId}</name>
 
   <properties>
     <karaf.localFeature>odl-ttp-model-rest</karaf.localFeature>
   </properties>
 
   <dependencies>
     <dependency>
       <groupId>org.apache.karaf.features</groupId>
       <artifactId>framework</artifactId>
       <type>kar</type>
     </dependency>
 
     <dependency>
       <groupId>org.opendaylight.ttp</groupId>
       <artifactId>features4-ttp</artifactId>
       <classifier>features</classifier>
       <version>${project.version}</version>
       <type>xml</type>
       <scope>runtime</scope>
     </dependency>
   </dependencies>
 </project>

(This is from TTP; replace the group and artifact identifiers, name, local feature and feature repository as appropriate.)

Step 8: add your karaf 4 features to distribution

Download int/dist repository:

git clone ssh://<user>@git.opendaylight.org:29418/integration/distribution

IMPORTANT: Make sure the projects your features depend on are already in the pom files (otherwise patch verification will fail):

  • artifacts/upstream/artifacts/pom.xml or artifacts/upstream/feature-repos/pom.xml
  • features/repos/index/pom.xml


1. Check your project version is correct in the file:

  • artifacts/upstream/properties/pom.xml


2. Declare karaf 4 feature repos and user-facing features dependencies:

If your project artifacts pom.xml contains your karaf 4 feature repos and features, just add your project artifacts dependency in:

  • artifacts/upstream/artifacts/pom.xml


Otherwise add your project karaf 4 feature repositories and your user-facing features to:

  • artifacts/upstream/feature-repos/pom.xml
  • artifacts/upstream/single-features/pom.xml


3. Add your karaf 4 feature repositories in:

  • features/repos/index/pom.xml


4. Add your user facing features in either:

  • features/singles/odl-integration-compatible-with-all/pom.xml if your feature is compatible.
  • features/singles/odl-integration-all/pom.xml if your feature is not compatible.


NOTE: You do not need to add a feature if it is already contained in another feature you have already included.

REMEMBER: Compatible features are those that can co-exist with other features in ODL because:

  • They do not interfere with any other feature
  • They are not network intrusive (e.g. configure network devices or push flows out-of-the-box)


Push changes to int/dist and wait for patch verification and review.

Caveats

<configfile>

Some feature contents can't be represented as Maven dependencies. They can all be handled using feature.xml stubs:

  • create a src/main/feature folder in your feature module
  • create a feature.xml file there, containing a feature stub with the same name as your feature, e.g.
<?xml version="1.0" encoding="UTF-8"?>
<features name="odl-mdsal-${project.version}" xmlns="http://karaf.apache.org/xmlns/features/v1.2.0">
    <feature name="odl-mdsal-xsql" version="${project.version}">
    </feature>
</features>
  • add the feature contents as before
<?xml version="1.0" encoding="UTF-8"?>
<features name="odl-mdsal-${project.version}" xmlns="http://karaf.apache.org/xmlns/features/v1.2.0">
    <feature name="odl-mdsal-xsql" version="${project.version}">
        <configfile finalname="${config.configfile.directory}/${config.xsql.configfile}">mvn:org.opendaylight.controller/sal-dom-xsql-config/${project.version}/xml/config</configfile>
    </feature>
</features>

The features will be merged into the final features file.

NB: The { {VERSION}} tag isn't available any more, you need to use Maven variable ${project.version} instead.

You also need to include the configfile in your POMs, as previously.

Beware that it now is feature/feature.xml and not features/features.xml ! (e.g. c/61339)

Karaf CLI commands shell.commands version issue

If you hit this problem:

Running org.opendaylight.odlparent.featuretest.SingleFeatureTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 15.955 sec <<< FAILURE! - in org.opendaylight.odlparent.featuretest.SingleFeatureTest
installFeature(org.opendaylight.odlparent.featuretest.SingleFeatureTest)[repoUrl: file:/home/vorburger/dev/ODL/git/genius/features/odl-genius/target/feature/feature.xml, Feature: odl-genius 0.2.0.SNAPSHOT]  Time elapsed: 15.917 sec  <<< ERROR!
org.osgi.service.resolver.ResolutionException: Unable to resolve root: missing requirement [root] osgi.identity; osgi.identity=odl-genius; type=karaf.feature; version="[0.2.0.SNAPSHOT,0.2.0.SNAPSHOT]"; filter:="(&(osgi.identity=odl-genius)(type=karaf.feature)(version>=0.2.0.SNAPSHOT)(version<=0.2.0.SNAPSHOT))" [caused by: Unable to resolve odl-genius/0.2.0.SNAPSHOT: missing requirement [odl-genius/0.2.0.SNAPSHOT] osgi.identity; osgi.identity=org.opendaylight.genius.itm-impl; type=osgi.bundle; version="[0.2.0.SNAPSHOT,0.2.0.SNAPSHOT]"; resolution:=mandatory [caused by: Unable to resolve org.opendaylight.genius.itm-impl/0.2.0.SNAPSHOT: missing requirement [org.opendaylight.genius.itm-impl/0.2.0.SNAPSHOT] osgi.wiring.package; filter:="(&(osgi.wiring.package=org.apache.karaf.shell.commands)(version>=3.0.0)(!(version>=4.0.0)))"]]

then you have to add this to the POM of the respective bundle defining Karaf CLI commands:

 <build>
   <plugins>
     <plugin>
       <groupId>org.apache.felix</groupId>
       <artifactId>maven-bundle-plugin</artifactId>
       <extensions>true</extensions>
       <configuration>
         <instructions>
           < ! -- This bundle works with Karaf 3 and 4.0, see https://wiki.opendaylight.org/view/Karaf_4_migration#Karaf_CLI_commands -- >
           <Import-Package>
             org.apache.karaf.shell.commands;version="[3.0.0,4.1)",
             org.apache.karaf.shell.console;version="[3.0.0,4.1)",
             *
           </Import-Package>
         </instructions>
       </configuration>
     </plugin>
   </plugins>
</build>

Karaf CLI commands IllegalArgumentException: Component must have a valid id

If you hit this problem:

diag failed; some bundles failed to start
diag: Failure {Starting=0, Failure=1, GracePeriod=5, Waiting=0, Installed=0, Resolved=4, Stopping=0, Unknown=0, Active=227}
1. NOK org.opendaylight.coe.cli: OSGi state = Active, Karaf bundleState = Failure, due to: Blueprint
Exception: Component must have a valid id java.lang.IllegalArgumentException: Component must have a valid id

at org.apache.aries.blueprint.parser.ComponentDefinitionRegistryImpl.registerComponentDefinition(ComponentDefinitionRegistryImpl.java:81)

possibly together with other Blueprint errors about Missing dependencies: objectClass=org.opendaylight.controller.md.sal.binding.api.DataBroker, objectClass=org.opendaylight.controller.md.sal.dom.spi.DOMNotificationSubscriptionListenerRegistry, objectClass=org.opendaylight.mdsal.binding.dom.codec.api.BindingNormalizedNodeSerializer, which seems to be just "fallout",

then you have to update the xmlns in the CLI shell bundle's Blueprint XML from http://karaf.apache.org/xmlns/shell/v1.0.0 to http://karaf.apache.org/xmlns/shell/v1.1.0", and remove the name= attribute on any <command> element, as done e.g. here https://git.opendaylight.org/gerrit/#/c/59256/16..18/northbound/cli/src/main/resources/org/opendaylight/blueprint/blueprint.xml.

Blueprint (BP)

If you hit this kind problem, related to missing trivial OSGi services registered by Blueprint (BP) which perfectly worked in Karaf 3 but fail now:

installFeature(org.opendaylight.odlparent.featuretest.SingleFeatureTest)[repoUrl: file:/home/vorburger/dev/ODL/git/genius/features/odl-genius/target/feature/feature.xml, Feature: odl-genius 0.2.0.SNAPSHOT]  Time elapsed: 14.392 sec  <<< ERROR!
org.osgi.service.resolver.ResolutionException: Unable to resolve root: missing requirement [root] osgi.identity; osgi.identity=odl-genius; type=karaf.feature; version="[0.2.0.SNAPSHOT,0.2.0.SNAPSHOT]"; filter:="(&(osgi.identity=odl-genius)(type=karaf.feature)(version>=0.2.0.SNAPSHOT)(version<=0.2.0.SNAPSHOT))" [caused by: Unable to resolve odl-genius/0.2.0.SNAPSHOT: missing requirement [odl-genius/0.2.0.SNAPSHOT] osgi.identity; osgi.identity=org.opendaylight.genius.idmanager-shell; type=osgi.bundle; version="[0.2.0.SNAPSHOT,0.2.0.SNAPSHOT]"; resolution:=mandatory [caused by: Unable to resolve org.opendaylight.genius.idmanager-shell/0.2.0.SNAPSHOT: missing requirement [org.opendaylight.genius.idmanager-shell/0.2.0.SNAPSHOT] osgi.service; effective:=active; filter:="(objectClass=org.opendaylight.genius.idmanager.api.IdManagerMonitor)"]]

then you may have to move src/main/resources/OSGI-INF/blueprint/ to src/main/resources/org/opendaylight/blueprint/, like in patch set 5 of change c/51966.

TODO This is... weird, surprising; so standard OSGi BP does not work in our tweaked Karaf - huh?!