Jump to: navigation, search

GettingStarted:Pulling, Hacking, and Pushing All the Code from the CLI

Intro

This page tries to give you a quick start guide to pulling, hacking, and pushing the code for an OpenDaylight project.

For a quick reference on some commonly used git commands, check out the GIT Cheat Sheet.

Make sure you have an account

While you can pull code anonymously, if you want to push it (or edit the wiki) you will need to have an OpenDaylight Account. To access the account creation form, currently you need to click the colorful icon here.

Gerrit Setup

Code reviews are enabled through Gerrit. Set up Gerrit for use if you wish to use ssh. If you wish to use https, Set up Gerrit to use HTTPS.

Note: You will need to perform the Gerrit Setup before you can access git via ssh as described below.

Install Dependencies

NOTE: If you have not setup your environment according to Development Environment Setup, make sure you at least follow Edit your .m2 settings.xml to ensure maven can resolve the correct dependencies.
NOTE: ODL now requires a minimum version of 3.3.9 for Boron+ releases! There are a significant number of dependencies required for building. The following works for Fedora 20 to add the dependencies:

  • Fedora
sudo yum -y install maven git bzip2 \
  make binutils gcc gcc-c++ boost-devel openssl-devel perl-ExtUtils-MakeMaker \
  unixODBC-devel gtest-devel redhat-lsb-core json-c-devel libcurl-devel \
  perl-Digest-SHA tomcat-native
  • Ubuntu 13.10/14.04 (assuming maven already installed using this guide)
   apt-get install pkg-config gcc make ant g++ git libboost-dev libcurl4-openssl-dev \
    libjson0-dev libssl-dev openjdk-7-jdk unixodbc-dev xmlstarlet

Pull code via git CLI

Pull the code by cloning the controller repository.

First, all the lines below use the $ODL_USERNAME environment variable, so your life will be *way* easier of you set it with:

export ODL_USERNAME=<username for the account you created at OpenDaylight>

or better yet, if you can use ssh (see below), add the following to your ~/.ssh/config file (create it if necessary):

Host git.opendaylight.org
    User <your username>

(then you can simply omit "${USERNAME}@" from the instructions below).

If your host is located behind a proxy that would drop ssh connections that uses ports other than http[s], you should add the following lines to ~/.ssh/config file

Host git.opendaylight.org
    User <your username>
    IdentityFile ~/.ssh/<public key, usually id_rsa.pub>
    ProxyCommand corkscrew <your proxy name or ip> <your proxy port, usually 8080 or 3128> %h %p
    Port 29418

This configuratioin requires a tool named corkscrew :

  • Fedora
sudo yum -y install corkscrew
  • Ubuntu 13.10/14.04
sudo apt-get install corkscrew

Then you will want to check out the code from each of the repos for each project. You can, of course select any one of the projects if you are only looking for one.

You have two choices for your method of engagement, you can use:

  1. ssh on port 29418 (blocked by some firewalls)
  2. https

Pull code via ssh

for PROJECT in aaa aalldp alto armoury atrium bgpcep bier capwap cardinal centinel coe controller daexim didm discovery \
dlux dluxapps docs eman faas federation fpc genius groupbasedpolicy honeycomb/vbd integration/distribution \
integration/packaging integration/packaging/ansible-opendaylight integration/packaging/puppet-opendaylight integration/test \
iotdm jsonrpc kafkaproducer l2switch lacp lispflowmapping mdsal messaging4transport natapp nemo netconf netide netvirt \
neutron next nic ocpplugin odlparent of-config ofextensions/circuitsw ofextensions/epc openflowjava openflowplugin opflex \
ovsdb packetcable persistence plugin2oc releng releng/autorelease releng/builder reservation sdninterfaceapp sfc snbi snmp \
snmp4sdn sxp systemmetrics tcpmd5 topoprocessing transportpce tsdr ttp unimgr usc usecplugin vpnservice vtn yang-push \
yangide yangtools; \
do git clone ssh://${ODL_USERNAME}@git.opendaylight.org:29418/${PROJECT}.git ${PROJECT}; \
done

Note: The best way to build an ODL distribution is to compile the integration/distribution project. The integration/distribution project pulls the latest artifacts published by each project and builds a karaf distribution where you can install and test out the latest features. There aren't many scenarios in which you would want to compile all of the projects on your own machine!

Pull code with https

for PROJECT in aaa aalldp alto armoury atrium bgpcep bier capwap cardinal centinel coe controller daexim didm discovery \
dlux dluxapps docs eman faas federation fpc genius groupbasedpolicy honeycomb/vbd integration/distribution \
integration/packaging integration/packaging/ansible-opendaylight integration/packaging/puppet-opendaylight integration/test \
iotdm jsonrpc kafkaproducer l2switch lacp lispflowmapping mdsal messaging4transport natapp nemo netconf netide netvirt \
neutron next nic ocpplugin odlparent of-config ofextensions/circuitsw ofextensions/epc openflowjava openflowplugin opflex \
ovsdb packetcable persistence plugin2oc releng releng/autorelease releng/builder reservation sdninterfaceapp sfc snbi snmp \
snmp4sdn sxp systemmetrics tcpmd5 topoprocessing transportpce tsdr ttp unimgr usc usecplugin vpnservice vtn yang-push \
yangide yangtools; \
do git clone https://git.opendaylight.org/gerrit/${PROJECT}.git ${PROJECT}; \
done

Checkout Stable branch

git checkout stable/<release name>

Checkout Release tag

git checkout tags/release/<release name>

Hack the Code

Build the code

FIXME: The process described below does not work for the Lithium SR2 release nor the latest master branch. Need to dig the correct instructions out of the builder and integration jobs and put here.

Please make sure you have followed the development environment setup guide properly and especially make sure that you have edited your ~/.m2/settings.xml.

Note that the code is structured by project, and each project has its own subdirectory. You must explicitly enter each subdirectory and build in that directory to build that project. In the example below, we show how to build ALL of the projects, but you can just build say the base controller by entering the controller directory and building there.

You are going to want to increase the memory available to maven, the settings on the Jenkins build server are:

export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m"

Below is the top level build line for each project:

for i in odlparent affinity bgpcep controller lispflowmapping openflowjava openflowplugin ovsdb/commons/parent vtn yangtools
do (cd "$i" && mvn clean install ); done

Note 1: You'll need at least 15 GiB of free space for the build to complete.

Note 2: If you are missing a build dependency, such as "make" the vtn build will fail, but likely report success.

Test the Code

Build and Run Karaf distribution

Without clearing .m2 cache (it contains built artifacts from last step), download integration code:

git clone ssh://${ODL_USERNAME}@git.opendaylight.org:29418/integration/distribution.git

or

git clone https://git.opendaylight.org/gerrit/p/integration/distribution.git

Build the integration code skipping test:

cd distribution
mvn clean install -Pq

Run the just created Karaf distribution:

cd distribution-karaf/target/assembly
bin/karaf

Install the features you want to test and perform any suitable test. You can find Karaf help on this link , including important tips for development under Karaf.

Gerrit Workflow

git-review is a command-line tool for Git / Gerrit. It makes it easier to configure your git clone, to submit a change or to fetch an existing one.

The upstream project is led by OpenStack. Not to be confused with the unrelated Facebook project.

Initial Setup

Clone your git repository, and in the root directory enter the following

git-review -s
git config user.name "Firstname Lastname"
git config user.email your_email@some.domain

This will set up your connection to Gerrit as well as your commit author and email. You can also pass the "--global" option if you want this to apply system wide rather then to the repo. For example:

git config --global user.name Firstname Lastname
git config --global user.email your_email@some.domain

Submitting Patches

The typical workflow when working on a new patch in Gerrit is to start off with a new branch representing the new feature or bug being fixed. This branch should only ever have exactly 1 commit off of origin/master and continuously "git commit --amend" to update the patch. A typical workflow might look like this:

git fetch origin
git checkout -b new_feature origin/master
git add <files>
git commit -s
gitk
  • We fetch origin to pull down the latest version of the code base to ensure that when branch with the most up to date code
  • The git checkout -b command creates a new branch called new_feature (you should name this appropriately)
  • Git add files you've modified to the staging area
  • Finally git commit with a relevant commit message the "-s" command signs off your commit message which is required to submit code to ODL.
  • Commit messages should be written in present-tense, imperative-style see examples of commit messages here https://wiki.openstack.org/wiki/GitCommitMessages
  • Use gitk to verify your changes are as you expect, it is good practice to check this often.

Note: To be accepted, all code must come with a developer certificate of origin as expressed by having a Signed-off-by. This means that you are asserting that you have made the change and you understand that the work was done as part of an open-source license.

Developer's Certificate of Origin 1.1

        By making a contribution to this project, I certify that:

        (a) The contribution was created in whole or in part by me and I
            have the right to submit it under the open source license
            indicated in the file; or

        (b) The contribution is based upon previous work that, to the best
            of my knowledge, is covered under an appropriate open source
            license and I have the right under that license to submit that
            work with modifications, whether created in whole or in part
            by me, under the same open source license (unless I am
            permitted to submit under a different license), as indicated
            in the file; or

        (c) The contribution was provided directly to me by some other
            person who certified (a), (b) or (c) and I have not modified
            it.

        (d) I understand and agree that this project and the contribution
            are public and that a record of the contribution (including all
            personal information I submit with it, including my sign-off) is
            maintained indefinitely and may be redistributed consistent with
            this project or the open source license(s) involved.

Mechanically you do it this way (long and short form):

git commit --signoff
git commit -s

If you wish to add the signoff to the commit message on your every commit without the need to specify -s or --signoff, add the following lines in the end of .git/hooks/commit-msg, after the line "add_ChangeId":

SOB=$(git var GIT_AUTHOR_IDENT | sed -n 's/^\(.*>\).*$/Signed-off-by: \1/p')
grep -qs "^$SOB" "$1" || echo "$SOB" >> "$1"

Once you've added and committed your changes, you can use git-review to publish to Gerrit after confirming your changes are up to date with upstream.

git fetch origin
git rebase origin/master
gitk
git review

The pattern git fetch, rebase, and gitk should be used often when you start working on a new change as well as just before you push your changes online, resolve any merge conflicts. If your branch is up to date you should be exactly 1 commit away from origin/master in the gitk graph view.

Git-review will rebase your changes against the master branch in Gerrit before pushing. It will also notify you of merge conflicts so you can manually resolve them. You can disable the rebase by using the -R or --no-rebase flags.

Adding a Bug

The last block of a commit message where your signed-off-by line is, is used by git as metadata fields. If you are working on something related to a bug you should add "Bug: 1234" to the commit message just above the Change-Id or Signed-off-by fields, it should be in the same block (no blank lines between them). If you forgot to do this when you committed you can amend your commit to add the metadata.

git commit --amend
# Add Bug: 1234 just above the Change-Id or Signed-off-by field

Adding a topic

By default, the Gerrit topic will be the name of your local branch. You can change this using the -t or --topic flags.

git review -t topic

Pushing to a feature branch

To push to a branch other than master, just add the branch name.

git review topic/schema
git review stable/hydrogen

Pushing a draft

Pushing to drafts is easy:

git review -D

or

git review --draft

Adding a dependency

To make your commit dependent upon another (parent) change...

# 
git review -d 6150
# Create a new branch called foo
git checkout -b foo
git commit 
git review -R

The -R option is there to avoid rebasing the parent change, as that would interfere with parent author view. Of course rebasing is encouraged if parent is already merged.

Updating a Patch (created from git remote brach)

Assuming you already have the local branch with your patch in, simply make your changes...

git commit --amend
git fetch origin
git rebase origin/master
gitk  # Verify your changes
git review

You should always follow the pattern fetch, rebase, and gitk to pull down the most recent code and rebase your code ontop of it and finally gitk verify your changes are as you expect.

Alternatively you can checkout a patch to review use the -d flag followed by the change number in Gerrit

git review -d 6150

Make your changes and resubmit to Gerrit.

git commit --amend
git review -t <original-topic> -f

-f will delete the local branch and checkout the master branch.

Getting Latest Dependent Change ("Rebase" without running rebase command)

In case the dependent change (parent-change) set was updated and you wish to "rebase" your change (child-change).

Fetch and checkout the parent change:

git review -d <parent-gerrit-id>

Cherry-pick your commit on top of it:

git review -x <child-gerrit-id>

Submit "rebased" change for review:

git review -R

Reviewing Patches

If you are downloading code to make changes or test functionality you can do so as follows:

git review -d 6150
git commit --amend
git review -t <original-topic> -f

If you wish to diff between two patch sets you can do so using git-review

git review -m 6150,3

Will diff between Patch Set 3 of change 6150 and the latest Patch Set.

Further Reading

The git-review wiki page is a great resource.

Appendix A: Why Gerrit Topics are Awesome!

Topics can be used as a search criteria in Gerrit by typing topic:foo. This allows commits that are part of a feature to be easily tracked.