Jump to: navigation, search

Simultaneous Release:Beryllium Release Plan

Contents

Introduction

This is a Simultaneous Release Plan for the Beryllium release of OpenDaylight. This is the fourth release of OpenDaylight and this release plan has substantial differences from it predecessors. Please read it carefully to understand the requirements to participate and for each milestone.

Projects may choose to participate or not based upon their readiness and desire to join the Simultaneous Release. This plan is structured as laid out in the OpenDaylight Project Lifecycle.

Definitions

APIs

  • APIs: For the purposes of being declared Stable, Provisional, or Tentative, an API is a collection of code that provides some high-level functionality, e.g., flow programming or MD-SAL data store access. When listed on a release plan, APIs should be given a short name, classified into one of the three categories, and have the supporting bundles (if/when they exist) listed as well.
    • Stable API: An API that can be accessed external to your project, existed in a previous version of ODL, and will continue to exist in the current version of ODL with no changes. By definition, all Stable APIs are frozen throughout this entire release cycle. Note that all APIs are assumed to be Stable APIs unless called out as Provisional and/or Tentative in a release plan.
    • Provisional API: An API that can be accessed external to your project and is introduced in the current release, or an externally accessible API that existed in a previous version of ODL but is being modified for the current release.
    • Tentative API: A Provisional API that will be provided in a best effort manner, but which may or may not be delivered in the final release. The Go/NoGo status of Tentative APIs must be made by M3.

Milestones

  • Functionality Freeze (M3): (used to be called feature freeze) No new externally visible functionality is to be added to the current release of ODL. All provisional APIs are at least functional (at a beta-quality level) if not yet fully tested.
  • API Freeze (M4): All externally accessible APIs (Stable and Provisional) may not be modified. An API exception process (similar to the one used in helium) will allow for critical changes to APIs after API freeze with the consent of the consuming projects. APIs include, but are not limited to, all Java classes/interfaces declared public and exported from an OSGi bundle, all YANG models, all Karaf feature names/locations, all config file YANG schemas, and all REST/RESTCONF calls including the full URL with options.
  • Code Freeze (M5): No new features/functionality are to be allowed into the current release. Only errors/bugs identified in the bugzilla system are allowed. The exceptions to this include new tests, and documentation. Distribution (Karaf) packaging must be complete. Errors/bugs found after Code Freeze are still bugs and they may be created and worked on. This includes packaging bugs found as well.
  • String Freeze (M5): All text strings used within ODL may not be changed. Final documentation and localization teams may rely on these strings not changing for the current release.
  • Release Candidate (RC): A fully-built, complete version of the current ODL release. This includes all code, documentation, and packaging that make up the final user-deliverable set of artifacts. After RC0, new RCs will be continually built, e.g., once per day, to enable rapid testing and fixing of bugs.
    • Note that this definition makes the dates for RCs and the final release as targets, but they may need to be adjusted based on project readiness and any remaining blocking bugs.
    • While we will build daily release candidates, the notion of RC#s (increasing in number at a longer cadence, e.g., weekly) will remain to aid in planning when bug fixes are expected.
    • During the RC process blocking bugs will be tracked both on a spreadsheet and in bugzilla.
    • During the RC process regular, e.g., daily, meetings will be held on IRC to identify and address critical issues as they arise.

Features

  • Feature: A logical grouping of code and functionality in a project. While a Feature is usually a Karaf feature, it could also be any other component or grouping.
  • Top-Level Feature: A Feature that provides one of the major pieces of functionality delivered by a project. In general, this should not require an understanding of the project's internals to know when/how to install it. Most projects will have a small number of, maybe even only one, Top-Level Features. In many cases the Top-Level Feature will actually only be a meta-feature grouping together lower-level features, which would be less obvious for an outsider to consume.
  • User-Facing Feature: A Top-Level Feature that somebody looking to install and run OpenDaylight should know about. They should be able to install and uninstall the Feature and should be able to tell that it's been installed in the form of new user interface elements, support for new southbound devices, or other mechanisms. For example, odl-mdsal-broker is not a user-facing feature as it does nothing on it's own. However, odl-mdsal-apidocs likely is as it generates a user interface. Features that don't provide explicit user interfaces can still be user-facing. For example, odl-bgpcep-bgp-all provides no explicit user interface, but does enable BGP support that a user might care about.

Stable and Extended Features

  • Stable Feature: A Top-Level Feature that meets the following criteria. The TSC is expected to review each potentially stable feature during the project's release review and ensure that it meets these requirements. Note: Any of these requirements can be waived by the TSC for a given feature. However, the TSC is encouraged to find a way to make the test fit the situation before granting a waiver.
    • The Feature must lie within the declared scope of the project.
    • The feature is part of a Mature project (as defined in the project lifecycle).
    • The feature was present in the previous release.
    • The feature is only dependent on other stable features.
    • The feature and it's dependencies must meet the criteria for being accepted into Maven Central.
    • The feature provides adequate documentation.
    • The feature has 75% or higher test coverage as reported by Sonar.
      • Note: You can find individual module/feature test coverage numbers by: (i) going to a sonar report, (ii) click on the percentage under "Overall Coverage", (iii) use the scrollable list on the left to find the relevant module/bundle. It helps if your feature's code is all under a single subdirectory and that there is nothing else in that directory.
    • All externally-facing, i.e., accessible from outside the JVM, APIs in the feature must be protected by community-accepted authentication/authorization mechanisms, e.g., using aaa-authn to protect RESTCONF APIs
    • The feature must have at least one automated test for each of:
      • functionality (previously called a system test) to show the basic functionality works
      • cluster compatibility to show the feature works in a 3-node cluster using the clustered data store
      • scalability to show how large a system, e.g., number of hosts, switches, or links, the feature can handle
      • performance to show how many operations, e.g., transactions, flows, linkstate events, per second the feature can handle
      • longevity/stability to show the feature can run for a period of time under load without ill effect
      • In each case, the tests must show no unexplained regressions from previous releases.
    • The feature is backward compatible with the previous release of the feature, e.g., any APIs that were not deprecated in the previous release still exist with the same signatures.
      • Note: This does not prohibit adding new functions, REST URLs, or data items, but typically would prohibit removing or changing existing such things.
    • The feature has no known vulnerabilities that are older than a week and classified as important by the security response team or high by their CVSS score in a CVE. If a fix for such a vulnerability lies outside of OpenDaylight, the TSC may choose to relax the requirement on a case-by-case basis.
    • The feature commits to providing a migration strategy from the previous release. This will ideally take the form of scripts or automatic upgrade support, but could also come in the forum of documentation.
  • Extended Feature: A top-level feature that is a part of the Beryllium release and does not meet the Stable Feature criteria.

Release Distributions

  • Beryllium Stable Distribution: A Karaf distribution containing the collection of Stable Features as they are compiled into the Beryllium Stable Release Feature Repository hosted in the Integration project.
  • Beryllium Extended Distribution: A Karaf distribution containing the collection of both Stable and Extended Features as they are compiled in the Beryllium Extended Release Feature Repository hosted in the Integration project.

Project Offsets

Projects are classified into one of 3 offsets:

  • offset zero: deadlines are at the dates listed
  • offset one: deadlines are generally1 at the listed dates + 2 weeks
  • offset two: deadlines are generally1 at the listed dates + 4 weeks
    • Note that the intent is that offset two projects have no other projects depending on them in this release

This is intentionally flattening the actual dependency graph

  • The full project-level graph is at least 10 levels
    • e.g., odlparent => yangtools => controller => openflowjava => openflowplugin => ovsdb => sfc => groupbasedpolicy => nic => integration
  • The idea is to hit an 80/20 point where projects can have some lag to get new APIs from those they depend on
    • If projects are in the same offset but need APIs from each other this should be noted and planned (possibly by asking for them sooner than would be required) as part of the API request negotiation at M2

The intent is for projects that form key infrastructure of all other projects (e.g., odlparent, yangtools, and controller) to operate at offset zero, projects which provide key network services (e.g., OpenFlow and OVSDB) to operate at offset one, and projects that others don't depend on to operate at offset two.

1Deadlines for Release Candidates (RC0, RC1 and RC2) and the release are the same regardless of offset. Deadlines for M1 and M2 are offset by +1 week and +2 weeks. Full details can be found in the dates listed in the Schedule table.

Requirements for Participation

In order to participate in the simultaneous release, a project must do the following things.

  1. Planning
    • Projects must declare their intent to participate in the Simultaneous Release by M1. This is accomplished by adding the project to the table in participating projects and sending the first milestone readout mail.
    • Participating projects must publish a candidate Release Plan by M1 and declare their final Release Plan by M2
      • Participating project Release Plans must contain Milestones that minimally line up with the Simultaneous Release Plan Milestones
      • Release plans should contain a complete list of the exposed APIs including the properties defined above, e.g., the name of the Java interface or YANG-file, a short name, and the list of supporting bundles.
      • Per-project release plans now include sections for cross-project negotiation of provided APIs and for noting cross-project incompatibilities.
        • Projects are required to negotiate cross-project dependencies for any new or modified APIs.
        • Projects are encouraged to think about and cross-project incompatibilities and how to resolve them, if possible, as part of their release plans.
  2. Leadership & Communication
    • Each project must elect a Project Lead as described in the TSC charter, section 7.
      • Phil Robb will help projects with this process and it must be completed by M1.
      • The results of the election, and other changes to the project lead during this release, should be reported by
        1. Updating the project facts template for the project on its main wiki page
        2. Updating the participating projects table of this release
        3. Sending an e-mail to the <project>-dev, release, and tsc lists
    • The Project Lead is expected to be responsible for the the project meeting deadlines, interacting with other projects, and interacting with the TSC
    • The Project Lead will be subscribed to the release mailing list and must respond to requests send to the a timely fashion—defined as two business days.
      • If a Project Lead is not be able to do so, they should (i) have somebody else stand in and do this on their behalf, (ii) send a mail to the release mailing list indicating this and the time period, and (iii) note the same information in the participating projects section of the release plan.
    • The project lead is expected to, at a minimum, read the release mailing list, read the TSC meeting minutes, and read the minutes from the IRC release meetings. The project lead is strongly encouraged to attend these meetings if at all possible and some representative from the project is expected to attend each IRC meeting if at all possible.
    • In addition to the Project Lead, each project must designate a Test Contact and Documentation Contact to handle test related communication.
    • All release-critical correspondence that requires a response will have a subject line containing "PLEASE RESPOND BY <DATE> <UTC-TIME>"
      • Please limit traffic to correspondence directly relating to the release
      • The TSC collects response time metrics for projects both to inform our planning and to measure project maturity going forward.
  3. Service Release Participation
    • All projects participating in the release are also required to participate in the stability releases described in the schedule after the formal release.
  4. Modularity
    • Modules that are not intended to interface with the controller via REST/other non-Java RPC mechanism must be OSGi bundles.
    • OSGi bundles should be reasonably granular.
    • OSGi bundles should be grouped into Karaf features by M3 including possibly defining some features as user-facing.
      • Each feature should be tested in every appropriate jenkins job (at least -verify, -merge, and -integration) using the "SingleFeatureTest" as defined in the Karaf step-by-step guide
  5. Quality
    • No later than M2, each project must have a "-verify" Jenkins Job which verifies that the project builds and passes test for each new patch pushed to gerrit.
    • No later than M2 as part of the Gerrit/Jenkins merge process, i.e., the Jenkins "-merge" job, participating projects must push their binary artifacts to the Nexus repository
    • No later than M2, each project must have a Jenkins Job which rebuilds and retests to an appropriate level when a project it depends on publishes new artifacts, i.e., a Jenkins "-integration" job.
    • No later than M2, each project primarily written in Java must be reporting unit and/or integration test coverage via sonar. (See instructions on reporting test code coverage)
      • Projects, especially ones that form key infrastructure for other projects, are strongly encouraged to set goals for code coverage and reported bugs. Doing so will be seen favorably when evaluating projects for advancement in the Project Lifecycle.
      • Stable Features must have appropriate unit and/or integration test coverage of at least 75% prior to M5.
  6. Testing
    • In addition to setting up appropriate Jenkins -verify, -merge, and -integration jobs by M2, projects are expected to provide adequate unit, integration and system tests.
      • Stable Features must have established integration and system tests as required for Mature project Stable Features.
    • The coverage provided by unit tests and integration tests should be reported to sonar by M2. (See instructions on reporting test code coverage)
    • Participating projects must describe a basic system test per top-level feature and a comprehensive system test including functionality, cluster, scalability, performance, longevity/stability per stable feature prior on M3.
    • Participating projects must run at least one basic automated system test for each top-level feature and several automated system tests including functionality, cluster, scalability, performance, longevity/stability for each stable feature by M5.
      • Note: The system test requirements can be waived by the TSC for a given feature if for example the top-level feature is tested through another top-level feature.
      • Note: Projects running system test outside OpenDaylight (external Lab) are required to report system test results in a timely fashion after release creations, e.g., weekly, RC, and formal releases.
    • Further details and requirements can be found in the schedule and Beryllium project integration and test requirements below.
  7. Documentation
    • Each participating project is expected to identify the kinds of documentation that would be useful (e.g., installation guide, user guide, developer guide) and provide them as part of the release.
    • More details on the expectations can be found in the schedule and Beryllium project documentation requirements below.
  8. Code Hygiene
    • No uses of System.out.println in non-test code.
    • No dependencies on 3rd party (non-ODL) snapshot versions
    • Willing to use agreed-upon versions for dependencies (both 3rd-party and ODL), i.e., to eliminate version skew
    • Willing to find source code for 3rd-party dependencies and/or move to versions or alternatives for which source code can be found.
  9. Distribution
    • All projects must support a Karaf-based distribution model including defining Karaf features and adding them to integration repository no later than M3.
    • No later than M3, each project must have a “-distribution" Jenkins Job to verify changes in the code do not break integration distribution.
  10. Meeting Deadlines
    • All projects are expected to meet the deadlines laid out in the schedule below.
      • To indicate this, the project lead/contact is expected to provide send a milestone readout to the release mailing list by 23:59:59 UTC on the date listed for the the appropriate offset at each milestone.
      • Most information will be communicated by filling out appropriate information in the release tracking spreadsheet, but a mail should still be sent indicating that the information has been filled in. Any other information or questions can be included in that mail.
    • If a project cannot make a deadline, the project lead/contact must write a summary of what was missed, why, the course correction that will be taken, and it's impact on other projects.
      • For offset two project this is mainly intended to be reflective and to help inform the release process.
      • For offset zero and offset one projects, this should be completed with 24 hours of missing the deadline and must be presented to the TSC at the first TSC meeting after the deadline.
    • All Milestone deliverables will be verified by the Beryllium release management staff and/or the TSC.
      • NOTE: For deliverables defined only in the project's release plan—and not as a requirement in this document—the release management staff and/or TSC will verify that the status of the deliverables has been reported. Beryllium release management staff and/or the TSC may also, but are not required to, verify the delivered functionality.


Beryllium Project Integration and Test Requirements

This section is excerpted from Beryllium Project Integration Requirements, to see it in it's full context, go to that page.

Kinds of Tests

These are "the 3 classics" in OpenDaylight:

  • Unit Test (UT)
    • This is the test developers write in the code to verify bundle functionality (e.g. JUnit).
    • This test exercises the contoller internal Java methods and classes.
    • Runs automatically on project build.
    • Test results and code coverage reported to Sonar.
  • Integration Test (IT)
    • This is the test developers write in the code to verify some inter-bundle (feature) functionality (e.g. PAX-EXAM).
    • This test exercises the controller internal Java interfaces.
    • Runs automatically on project build.
    • Test results and code coverage reported to Sonar.
    • For more information about IT check Integration Test wiki .
  • System Test (ST)
    • This is the test to verify multiple features at system level (e.g. Robot FW).
    • This test exercises the controller external REST interface and plugin protocols.
    • Runs on a test bed made of robot, controller and device instances.
    • Requires a karaf distribution as System Under Test (SUT).
    • Test results reported in Jenkins at a minimum.
    • We strongly recommend the use of Robot Framework for system test (ST), we already have extensive knowledge, examples and tutorials.
    • For more information about ST check System Test Step by Step Guide.

Test Strategy

The CI process includes 3 key events:

  • Code Submit ->
    • Upon code submit project verify job will build the project using the submitted patch and so it will automatically run Unit Test (UT) and Integration Test (IT).
    • We can optionally trigger System Test (ST) upon code commit, this would be useful on sensitive projects that can break others. For more information check the Patch Test Job.
  • Code Merge ->
    • Upon code merge, project merge job will build the project and so it will automatically run Unit Test (UT), Integration Test (IT) and upload artifacts to Nexus.
    • After the project merge job is completed, the project distribution job will build the integration distribution to make sure new artifacts do not break the karaf distribution.
    • The project System Test (ST) will run after the new karaf distribution is created and uploaded to Nexus.
    • Downstream projects will trigger their own build (integration job) after upstream code merge and System Test (ST) after an upstream distribution creation
  • Release Creation ->
    • Upon release creation, a distribution test job will be triggered to pass all available system test (ST).

We also have periodic tests:

  • Daily ->
    • Some tests (scalability, longevity, platform test, etc…) are too long to run in CI (every patch) so we run them once every day. As a rule of thumb a test surpassing 10 minutes should be split in shorter tests or run in a daily fashion.

Requirements for Projects

In this release projects are expected to:

  • Designate a project Test Contact for all test questions and matters.
  • Setup the Continous Integration process according to Project Checklist.
  • Provide adequate unit and integration tests for top-level features as well as fulfill quality requirements for stable features (i.e. unit and/or integration test coverage of at least 75%)
  • Define project karaf features and include them in the integration Karaf distribution. More information in Karaf Step by Step Guide.
  • Have a distribution job to verify changes do not impact the integration distribution (this will be automatically setup by the releng/builder group).
  • Describe a basic system test per top-level feature and a comprehensive system test including functionality, cluster, scalability, performance, longevity/stability per stable feature.
    • Note: Integration group will provide templates and examples for this.
  • Run at least one basic automated system test job for each top-level feature. More information in System Test Step by Step Guide.
  • Run several automated system test jobs including functionality, cluster, scalability, performance, longevity/stability for each stable feature.
    • Note: The system test requirements can be waived by the TSC for a given feature if the top-level feature is tested through another top-level feature for example.
    • Note: Projects can also run the system test outside OpenDaylight (external Lab). In this case they are required to report system test results in a timely fashion after release creations, e.g., weekly, RC, and formal releases.

Timeline for Deliverables

  • M1: Joining the Release
    • Projects must designate a Test Contact.
  • M2: Final Release Plan
    • Projects must setup the Continous Integration process according to Project Checklist.
    • Projects may apply for a system test waiver if they think they have top-level features not requiring system test or covered by other top-level features test.
    • Projects must specify whether they plan to use OpenDaylight CI infrastructure for system test. It is recommended to use the OpenDaylight CI infrastructure unless there is some HW or SW resource that cannot be installed there. Projects running system test in external Labs are required to report system test results in a timely fashion after release creations, e.g., weekly, RC, and formal releases.
  • M3: Feature Test Started
    • Projects must define karaf features and include them in the integration karaf distribution. More information in Karaf Step by Step Guide.
    • Projects must have a distribution job to verify changes do not impact the integration distribution (this will be automatically setup by the releng/builder group).
    • Projects must have filled out a basic system test plan template for each top-level feature (karaf and not karaf) and a comprehensive system test template including functionality, cluster, scalability, performance, longevity/stability per stable feature.
  • M4: Feature Test Continues
    • Projects must have all extra SW configuration and resources required for system test installed in the OpenDaylight CI. More information in How To Install SW in CI.
  • M5: Feature Test Complete
    • Stable features should fulfill quality requirements (i.e. unit and/or integration test coverage of at least 75%)
    • Projects must run at least one basic automated system test job for each top-level feature. More information in System Test Step by Step Guide.
    • Projects must run several automated system test jobs including functionality, cluster, scalability, performance, longevity/stability for each stable feature.


Beryllium Project Documentation Requirements

This section is excerpted from Beryllium Project Documentation Requirements, to see it in it's full context, go to that page.

Kinds of Documentation

These are the expected kinds of documentation and target audiences for each kind.

  • User/Operator: for people looking to use the feature w/o writing code
    • Should include an overview of the project/feature
    • Should include description of availbe configuration options and what they do
  • Developer: for people looking to use the feature in code w/o modifying it
    • Should include API documentation, e.g., enunciate for REST, Javadoc for Java, ??? for RESTCONF/models
  • Contributor: for people looking to extend or modify the feature's source code
    • Note: should be documented on the wiki not in asciidoc
  • Installation: for people looking for instructions to install the feature after they have downloaded the ODL release
    • Note: audience is the same as User/Operator docs
    • For most projects, this will be just a list of top-level features and options
      • As an example, l2switch-switch as the top-level feature with the -rest and -ui options
      • We'd also like them to note if the options should be checkboxes (i.e., they can each be turned on/off independently) or a drop down (i.e., at most one can be selected)
      • What other top-level features in the release are incompatible with each feature
      • This will likely be presented as a table in the documentation and the data will likely also be consumed by automated installers/configurators/downloaders
    • For some projects, there is extra installation instructions (for external components) and/or configuration
      • In that case, there will be a (sub)section in the documentation describing this process.
  • HowTo/Tutorial: walk throughs and examples that are not general-purpose documentation
    • Generally, these should be done as a (sub)section of either user/operator or developer documentation.
    • If they are especially long or complex, they may belong on their own
  • Release Notes:
    • Release notes are required as part of each project's release review. They must also be translated into AsciiDoc for inclusion in the formal documentation.

Requirements for projects

Projects MUST do the following

  • Provide AsciiDoc-format documentation including
    • Developer documentation for every feature
      • Most projects will want to logically nest the documentation for individual features under a single project-wide chapter or section
      • This can be provided as a single .adoc file or multiple .adoc files if the features fall into different groups
      • This should start with ~300 word overview of the project and include references to any automatically-generated API documentation as well as more general developer information (as described above).
    • User/Operator documentation for every every user-facing feature (if any)
      • Note: This should be per-feature, not per-project. User's shouldn't have to know which project a feature came from.
      • Intimately related features, e.g., l2switch-switch, l2switch-switch-rest, and l2switch-switch-ui, can be documented as one noting the differences
      • This can be provided as a single .adoc file or multiple .adoc files if the features fall into different groups
    • Installation documentation
      • Most projects will simply provide a list of user-facing features and options. See Kinds of Documentation above.
    • Release Notes (both on the wiki and AsciiDoc) as part of the release review.
  • This documentation will be contributed to the docs repo (or possibly imported from the project's own repo with tooling that is under development)
    • Projects MAY be ENCOURGAGED to instead provide this from their own repository if the tooling is developed
    • Projects choosing to meet the requirement this way MUST provide a patch to docs repo to import the project's documentation
  • Projects MUST cooperate with the documentation group on edits and enhancements to documentation

Timeline for Deliverables from Projects

  • M3: Documentation Started
    • Identified the kinds of documentation that will be provided and for what features
      • Release Notes are not required until release reviews at RC2
    • Created the appropriate .adoc files in the docs repository (or their own repository if the tooling is available)
    • Have an outline for the expected documentation in those .adoc files including the relevant (sub)sections and a sentence or two explaining what will go there
      • Obviusly, providing actual documentation in the (sub)sections is encouraged and meets this requirement
    • Milestone readout should include
      1. the list of kinds of documentation
      2. the list of corresponding .adoc files and their location, e.g., repo and path
      3. the list of commits creating those .adoc files
      4. the current word counts of those .adoc files
  • M4: Documentation Continues
    • The readout at M4 should include the word counts of all .adoc files with links to commits
    • The goal is to have draft documentation complete so that the documentation group can comment on it.
  • M5: Documentation Complete
    • All (sub)sections in all .adoc files have complete, readable, usable content.
    • Ideally, there should have been some interaction with the documentation group about any suggested edits and enhancements
  • RC2: Release notes
    • Projects must provide release notes as .adoc pushed to integration (or locally in the project's repository if the tooling is developed)

Milestones, Release Candidates, and Service Releases

  • Milestones are spaced roughly 4 weeks apart taking into account significant holidays.
  • Release Candidates (RC) are spaced 2 weeks apart
  • Service Releases are roughly 6, 12, 24, and 36 weeks after the Formal Beryllium Release and are intended to continue at least until the after the next formal release of the OpenDaylight, presumably Boron.

Schedule Framework

This Simultaneous Release plan has been drafted based on the Schedule Framework

Schedule

2The deadline to meet and report the results of each milestone is at 23:59:59 UTC on the listed day. That corresponds to 4p or 5p pacific time.

Milestone Offset 0 Date2 Offset 1 Date2 Offset 2 Date2 Events
M0 7/16/2015 N/A N/A
  • Beryllium Simultaneous Release Open
  • Note: the date for M0 will be at least one day after the TSC approves the Beryllium release plan.
Last date for project proposals 7/9/2015 7/16/2015 7/23/2015

This is the latest date a project proposal can be sent to the project-proposals list and still have the required two week public comment period before its project creation review at the last TSC meeting before the M1 milestone. Project proposals submitted after this date will not be able to become formal projects by M1 and thus will not be able to participate in the Beryllium release.3

M1 7/23/2015 7/30/2015 8/6/2015
  1. Projects must have declared intent to participate in Simultaneous Release
  2. Projects must have elected their Project Leads and specify a Test Contact
  3. Participating Projects must have published a candidate Release Plan for public comment ( Release Plan Template )
    • Note that the release plan includes details about negotiating inter-project dependencies, expectations, and incompatibilities.
M2 8/20/2015 8/27/2015 9/3/2015
  1. Participating Projects must have declared their final Release Plan with all sections fully completed.
  2. Projects that need extra configuration or resources other than those availble in the OpenDaylight CI infrastructure must have opened helpdesk tickets to add them.
  3. Project Checklist completed (for all projects, not just new ones).
  4. Projects may apply for a system test waiver if they think they have top-level features not requiring system test or covered by other top-level features test.
  5. Projects must specify whether they plan to use OpenDaylight CI infrastructure for system test. It is recommended to use the OpenDaylight CI infrastructure unless there is some HW or SW resource that cannot be installed there. Projects running system test in external Labs are required to report system test results in a timely fashion after release creations, e.g., weekly, RC, and formal releases.
  6. Project must get acknowledged from all projects that it depends on.
M3 9/17/2015 10/1/2015 10/15/2015
  1. Feature/Functionality Freeze
    • Final list of externally consumable APIs defined and documented
      • Projects must state for each TENTATIVE API they have (if any) whether they are formally planning to deliver it.
        • If so, it should be noted that it will be delivered.
        • If not projects requesting the API must be informed so that they can take corrective actions.
      • Externally consumable APIs are available at beta-quality
    • All inter-project dependencies are resolved (all project functionality is declared as either "In" or "Out" of this release)
  2. Karaf Features defined
    • Instructions can be found in the Karaf:Step by Step Guide
      • Each feature should be tested in every appropriate jenkins job (at least -verify, -merge, and -integration) using the "SingleFeatureTest" as defined in the Karaf step-by-step guide
    • Any feature repositories containing features intended for release must be added to the main features.xml file in the integration git repository as explained in the Karaf step-by-step guide
      • Projects must have a distribution job to verify changes in code do not impact the integration distribution (this will be automatically setup by the releng/builder group).
    • Features that are intended to be "top-level", "user-facing" and/or "stable" must be called out in the milestone readout. These features will have additional requirements:
    • Changing the name of a Karaf feature or removing a Karaf feature should be handled via an API freeze waiver after this point
  3. Documentation Started
    • Identified the kinds of documentation to be provided, created AsciiDoc files for them with outlines, and committed those files in an appropriate location. (See the documentation requirements section above for more details.)
  4. Feature Test Started
M4 10/15/2015 11/12/2015 12/3/2015
  1. API Freeze: See more information in the definition above.
  2. Documentation: Project readouts MUST include a word count of each relevant .adoc file with a goal of draft documentation done.
  3. Projects are encouraged to meet the requirements to be included in maven central
    • Project readout MUST include whether or not this was accomplished
  4. Feature Test Continues
    • Participating projects Projects must have all extra SW configuration and resources required for system test installed in the OpenDaylight CI4. More information in How To Install SW in CI.
M5 12/3/2015 12/17/2015 1/14/2016
  1. Code Freeze (bug fixes only from here as defined above)
  2. Stability branch, i.e., stable/beryllium, must be cut and local project versions bumped on master to avoid overwriting Beryllium SNAPSHOTS
  3. String Freeze (all externally visible strings frozen to allow for translation & documentation)
  4. Documentation Complete: Only editing and and enhancing should take place after this point.
  5. Feature Test Complete
    • Stable features should fulfill quality requirements listed in definitions section
    • Projects must run at least one basic automated system test job for each top-level feature and several automated system test jobs including functionality, cluster, scalability, performance, longevity/stability for each stable feature4.
RC0 1/21/2016 N/A N/A
  1. The build for RC0 will start at 23:59:59 UTC on 1/21/2016
  2. During the RC process, regular, e.g., daily, IRC meetings will take place to identify and address issues
  3. During the RC process, blocking bugs will be tracked in bugzilla and a common spreadsheet
RC1 1/28/2016 N/A N/A
  1. The build for RC1 will start at 23:59:59 UTC on 1/28/2016
  2. During the RC process, regular, e.g., daily, IRC meetings will take place to identify and address issues
  3. During the RC process, blocking bugs will be tracked in bugzilla and a common spreadsheet
RC2 2/4/2016 N/A N/A
  1. The build for RC2 will start at 23:59:59 UTC on 2/4/2016
  2. During the RC process, regular, e.g., daily, IRC meetings will take place to identify and address issues
  3. During the RC process, blocking bugs will be tracked in bugzilla and a common spreadsheet
RC3 2/11/2016 N/A N/A
  1. Participating Projects must hold their Release Reviews, including User Facing Documentation.
  2. The version suffix for RC3 will be -Beryllium instead of -Beryllium-RCX or -Beryllium-RCX-<timestamp> so that the build can be released if it is found to be free of blocking bugs.
  3. The build for RC3 will start at 23:59:59 UTC on 2/11/2016
  4. During the RC process, regular, e.g., daily, IRC meetings will take place to identify and address issues
  5. During the RC process, blocking bugs will be tracked in bugzilla and a common spreadsheet
Formal Beryllium Release 2/18/2016 N/A N/A
  1. Formal Beryllium Release
    • NOTE: The build to produce the formal release artifacts is likely to occur before 2/18/2016.
  2. After the release, projects MUST apply the release patch to the stable/beryllium branch and bump versions. Unless a project opts out, this will be done automatically by the release team after the release.
    • Note: Any patches merged to stable/beryllium after the auto-release build that produces the formal release artifacts, but before the release patch and version bumps are applied will have to be reverted and re-applied after the release and version bump patches.
SR1 (Service Release 1 aka Beryllium.1) 3/17/2016 N/A N/A
  1. First Service Release for Beryllium. NOTE: This date is provisional, but will not move earlier. Please note, event based Updates (security/critical bugs) are distinct and may occur at any point.
    • To allow time for testing, a release candidate will be built before the service release and projects are expected to not merge patches except for blocking bugs between that time and the actual service release.
    • Blocking bugs will be tracked via bugzilla and a spreadsheet.
  2. After the release, projects MUST apply the release patch to the stable/beryllium branch and bump versions. Unless a project opts out, this will be done automatically by the release team after the release.
    • Note: Any patches merged to stable/beryllium after the auto-release build that produces the formal release artifacts, but before the release patch and version bumps are applied will have to be reverted and re-applied after the release and version bump patches.
SR2 (Service Release 2 aka Beryllium.2) 4/28/2016 N/A N/A
  1. Second Service Release for Beryllium. NOTE: This date is provisional, but will not move earlier. Please note, event based Updates (security/critical bugs) are distinct and may occur at any point.
    • To allow time for testing, a release candidate will be built before the service release and projects are expected to not merge patches except for blocking bugs between that time and the actual service release.
    • Blocking bugs will be tracked via bugzilla and a spreadsheet.
  2. After the release, projects MUST apply the release patch to the stable/beryllium branch and bump versions. Unless a project opts out, this will be done automatically by the release team after the release.
    • Note: Any patches merged to stable/beryllium after the auto-release build that produces the formal release artifacts, but before the release patch and version bumps are applied will have to be reverted and re-applied after the release and version bump patches.
SR3 (Service Release 3 aka Beryllium.3) 7/28/2016 N/A N/A
  1. Third Service Release for Beryllium. NOTE: This date is provisional, but will not move earlier. Please note, event based Updates (security/critical bugs) are distinct and may occur at any point.
    • To allow time for testing, a release candidate will be built before the service release and projects are expected to not merge patches except for blocking bugs between that time and the actual service release.
    • Blocking bugs will be tracked via bugzilla and a spreadsheet.
  2. After the release, projects MUST apply the release patch to the stable/beryllium branch and bump versions. Unless a project opts out, this will be done automatically by the release team after the release.
    • Note: Any patches merged to stable/beryllium after the auto-release build that produces the formal release artifacts, but before the release patch and version bumps are applied will have to be reverted and re-applied after the release and version bump patches.
SR4 (Service Release 4 aka Beryllium.4) 10/20/2016 N/A N/A
  1. Fourth Service Release for Beryllium. NOTE: This date is provisional, but will not move earlier. Please note, event based Updates (security/critical bugs) are distinct and may occur at any point.
    • To allow time for testing, a release candidate will be built before the service release and projects are expected to not merge patches except for blocking bugs between that time and the actual service release.
    • Blocking bugs will be tracked via bugzilla and a spreadsheet.
  2. After the release, projects MUST apply the release patch to the stable/beryllium branch and bump versions. Unless a project opts out, this will be done automatically by the release team after the release.
    • Note: Any patches merged to stable/beryllium after the auto-release build that produces the formal release artifacts, but before the release patch and version bumps are applied will have to be reverted and re-applied after the release and version bump patches.

3Please note that the TSC reserves the right to allow projects to enter the Simultaneous Release for a reasonable period of time after the M1 date. For example, the TSC may allow additional time if a project is delayed by the IPR Review process.

4Projects running system tests outside the OpenDaylight CI infrastructure are not required to run system tests and report the results on "-merge" and "-integration" Jenkins jobs, although if they can this is ideal. They are required to report system test results in a timely fashion after release creations, e.g., weekly, RC, and formal releases.

Please also note that projects that would like to spin out parts of themselves into additional projects may have those new projects join the Simultaneous Release at any point prior to M3 provided:

  1. The TSC has been informed of this intent prior to M3
  2. The original project's release Release Plan is apportioned between the original and new projects with no parts missing
  3. The new projects have been proposed and approved by the TSC into one of the non-proposed life-cycle states in the normal manner by M3
  4. The new projects have completed the requirements for all milestones before they joined the release, e.g., M1 and/or M2

Lastly, note that as the new projects are joining the release prior to M3, they must meet all the requirements for M3 at the normal time.

Participating Projects

The list of participating projects can be found in this spreadsheet: https://docs.google.com/spreadsheets/d/1Kfp5HVQGydNegEjp6dD2V3XsUnqpice0bysWbdxABA4/edit#gid=1580006349

Note: It is not embedded because it causes browsers to behave badly when it is.

Participating projects should list themselves here prior to M1, with a link to their Project wiki page and their Release Plan.

Offset 0 Projects

Project Name PTL Name PTL Email Test Contact Name Test Contact Email Doc Contact Name Doc Contact Email
AAA Wojciech Dec wdec@cisco.com Carmen Kelling carmen.w.kelling@hp.com Wojciech Dec wdec@cisco.com
Controller Tony Tkacik ttkacik@cisco.com Tony Tkacik ttkacik@cisco.com Tony Tkacik ttkacik@cisco.com
MD-SAL Tony Tkacik ttkacik@cisco.com Tony Tkacik ttkacik@cisco.com Tony Tkacik ttkacik@cisco.com
NETCONF Maros Marsalek mmarsale@cisco.com Maros Marsalek mmarsale@cisco.com Maros Marsalek mmarsale@cisco.com
odlparent Stephen Kitt skitt@redhat.com Stephen Kitt skitt@redhat.com Stephen Kitt skitt@redhat.com
YANG Tools Robert Varga rovarga@cisco.com Tony Tkacik ttkacik@cisco.com Tony Tkacik ttkacik@cisco.com

Offset 1 Projects

Project Name PTL Name PTL Email Test Contact Name Test Contact Email Doc Contact Name Doc Contact Email
BGPCEP Milos Fabian milfabia@cisco.com Milos fabian milfabia@cisco.com Milos Fabian milfabia@cisco.com
DLUX Harman Singh harmasin@cisco.com Harman Singh harmasin@cisco.com Harman Singh harmasin@cisco.com
L2Switch Amit Mandke ammandke@cisco.com Amit Mandke ammandke@cisco.com Amit Mandke ammandke@cisco.com
LispFlowMapping Vina Ermagan vermagan@cisco.com Lorand Jakab lojakab@cisco.com Vina Ermagan vermagan@cisco.com
Neutron Northbound (NN) Ryan Moats rmoats@us.ibm.com Ankur Gupta ankur.gupta@intel.com Ryan Moas rmoats@us.ibm.com
Openflow Abhijit Kumbhare abhijitkoss@gmail.com Jamo Luhrsen jluhrsen@gmail.com Abhijit Kumbhare abhijitkoss@gmail.com
Openflowjava Michal Polkorab michal.polkorab@pantheon.sk Michal Polkorab michal.polkorab@pantheon.sk Michal Polkorab michal.polkorab@pantheon.sk
SNMP Adam Dierkens adam@dierkens.com Vaishali Mithbaokar vmithbao@cisco.com Vaishali Mithbaokar vmithbao@cisco.com
SNMP4SDN Yi-Ling Hsieh (Christine) ylhsieh@itri.org.tw Yi-Ling Hsieh (Christine) ylhsieh@itri.org.tw Yi-Ling Hsieh (Christine) ylhsieh@itri.org.tw
SXP Matthew Robertson mrobertson@lancope.com Matthew Robertson mrobertson@lancope.com Matthew Robertson mrobertson@lancope.com
TCPMD5 Robert Varga rovarga@cisco.com Milos Fabian milfabia@cisco.com Milos Fabian milfabia@cisco.com
Topoprocessing Andrej Zan andrej.zan@pantheon.sk Andrej Zan andrej.zan@pantheon.sk Andrej Zan andrej.zan@pantheon.sk
Unified Secure Channel (USC) An Ho an.ho@huawei.com An Ho an.ho@huawei.com An Ho an.ho@huawei.com

Offset 2 Projects

Project Name PTL Name PTL Email Test Contact Name Test Contact Email Doc Contact Name Doc Contact Email
ALTO Kai Gao/Shu Dong gaok12@mails.tsinghua.edu.cn Jensen Zhang jingxuan.n.zhang@gmail.com Yichen Qian 92yichenqian@tongji.edu.cn
Armoury Alexis de Talhouet adetalhouet@inocybe.com Alexis de Talhouët adetalhouet@inocybe.com Subhash Kumar Singh subh.singh007@gmail.com
CAPWAP Navin Agrawal anavin@gmail.com Hafis Saly hsaly@Brocade.com Navin Agrawal anavin@gmail.com
Centinel Sumit Kapoor sumit.kapoor@tcs.com Abhishek Abhi abhishek.abhi@tcs.com Sunaina khanna sunaina.khanna@tcs.com
Controller Shield (CS) Thomas Lee S thomaslee.s@tcs.com Rafat Jahan rafat.jahan@tcs.com Rafat Jahan rafat.jahan@tcs.com
DIDM Anandhi Manikantan anandhi.manikantan@hpe.com Linh Pham linh.pham@hpe.com Anandhi Manikantan anandhi.manikantan@hpe.com
Documentation Colin Dixon colin@colindixon.com Colin Dixon colin@colindixon.com N/A N/A
Fabric as a Service (FaaS) Xingjun Chu Xingjun.Chu@huawei.com feng dong leslin.dongfeng@huawei.com david song songwei80@huawei.com
Group Base Policy Keith Burns alagalah@gmail.com Tomas Cechvala tcechval@cisco.com Keith Burns alagalah@gmail.com
Integration/Distribution Luis Gomez ecelgp@gmail.com Luis Gomez ecelgp@gmail.com Luis Gomez ecelgp@gmail.com
IoTDM John Burns johnburn@cisoc.com Cangji Wu canwu@cisco.com John Burns johnburn@cisco.com
LACP Venkataraghavan Chetput C_Venkataraghavan@dell.com Venkataraghavan Chetput C_Venkataraghavan@dell.com Venkataraghavan Chetput C_Venkataraghavan@dell.com
Messaging4Transport Pradeeban Kathiravelu kk.pradeeban@gmail.com Pradeeban Kathiravelu kk.pradeeban@gmail.com Pradeeban Kathiravelu kk.pradeeban@gmail.com
NEMO Tianran Zhou zhoutianran@huawei.com Tianran Zhou zhoutianran@huawei.com Tianran Zhou zhoutianran@huawei.com
NetIDE Alec Leckey alexander.j.leckey@intel.com Alec Leckey alexander.j.leckey@intel.com Alec Leckey alexander.j.leckey@intel.com
Network Intent Composition (NIC) Raphael Amorim raphael.amorim@hpe.com Carmen Kelling carmen.w.kelling@hpe.com Raphael Amorim raphael.amorim@hpe.com
NeXt UI Toolkit Aikepaer Abuduweili aaikepae@cisco.com Aikepaer Abuduweili aaikepae@cisco.com Aikepaer aaikepae@cisco.com
ODL-SDNi Shahid Shaik shahid.b@tcs.com Divya kuppili.dcharitha@tcsin.com Karteek karteek.koraganji@tcs.com
of-config Wei Meng meng.wei2@zte.com.cn Rui Hu hu.rui2@zte.com.cn Susan Xu xu.hui7@zte.com.cn
OpFlex Robert Adams readams@readams.net Rob Adams readams@readams.net Rob Adams readams@readams.net
OVSDB Sam Hague shague86@gmail.com Alexis De Talhouet adetalhouet@inocybe.com Andre Fredette afredette@redhat.com
PacketCable PCMM Brian Otte b.otte@cablelabs.com Brian Otte b.otte@cablelabs.com Brian Otte b.otte@cablelabs.com
Releng/Autorelease Thanh Ha thanh.ha@linuxfoundation.org Thanh Ha thanh.ha@linuxfoundation.org Thanh Ha thanh.ha@linuxfoundation.org
Reservation Mathieu Lemay mlemay@inocybe.com Alexis De Talhouet adetalhouet@inocybe.com Gabriel Robitaille-Montpetit grmontpetit@inocybe.com
SFC Brady Johnson brady.allen.johnson@ericsson.com Johnson Li johnson.li@intel.com Brady Johnson brady.allen.johnson@ericsson.com
SNBI FrankBrockners fbrockne@cisco.com Vijay Anand R vanandr@cisco.com Vijay Anand R rambalas@cisco.com
TSDR YuLing Chen yuling_c@dell.com Vasanthan Balasubram Vasanthan_Balasubram@Dell.com YuLing Chen yuling_c@dell.com
TTP Curt Beckmann beckmann@brocade.com Colin Dixon colin@colindixon.com Curt Beckmann beckmann@brocade.com
UNI Manager Kevin Luehrs k.luehrs@cablelabs.com Mufaddal Makati m.makati@cablelabs.com Kevin Luehrs k.luehrs@cablelabs.com
VPNService Prem Sankar prem.sankar.g@ericsson.com Sindhuri B sindhuri.b@ericsson.com Prem Sankar prem.sankar.g@ericsson.com
VTN Hideyuki Tai Hideyuki.Tai@necam.com Venkatrangan Govindarajan venkatrangang@hcl.com Venkatrangan Govindarajan venkatrangang@hcl.com
YANG Pub/Sub Ambika Prasad Tripathy ambtripa@cisco.com Jayshankar T jayat@cisco.com Swaroop Shastri shastriswaroop@hotmail.com

Project Status

Project Status


Beryllium Download

Beryllium SR1 Download

Beryllium SR2 Download

Beryllium SR3 Download

Beryllium SR4 Download

Release Reviews

  • Release Review Status
  • 05-Feb-2016
    • USECPLUGIN: Approved (Conditional)
  • 08-Feb-2016
    • USC: Approved
    • NEMO: Approved
    • CAPWAP: Approved
  • 09-Feb-2016
    • ODLPARENT: Approved
    • YANGTOOLS: Approved
    • AAA: Approved
    • CONTROLLER: Approved
    • MDSAL: Approved
    • NETCONF: Approved
    • BGPCEP: Approved
    • L2SWITCH: Approved
    • OPENFLOWJAVA: Approved
  • 11-Feb-2016
    • NEUTRON: Approved
    • SDNINTERFACEAPP: Pending (Follow Up)
    • SXP: Approved
    • TCPMD5: Approved
    • YANGPUSH: Approved
    • ALTO: Approved
    • DIDM: Approved
    • VTN: Approved
    • DLUX: Approved
    • FAAS: Approved (Conditional)
    • LISPFLOWMAPPING: Approved
  • 12-Feb-2016
    • SNBI: Approved (Conditional)
    • IOTDM: Approved
    • LACP: Approved
    • NETIDE: Approved
    • NIC: Approved
    • PACKETCABLE: Approved
    • VPNSERVICE: Approved
    • MESSAGING4TRANSPORT: Approved
    • OPENFLOWPLUGIN: Approved
    • OPFLEX: Approved
  • 15-Feb-2016
    • SFC: Approved
    • TTP: Approved
    • OVSDB: Approved
    • SNMP4SDN: Approved
    • TOPOPROCESSING: Approved
    • UNIMGR: PENDING (Follow Up)
    • CENTINEL: PENDING (Follow Up)
    • TSDR: Approved
    • SNMP: Approved
    • GROUPBASEDPOLICY: Approved
  • By Email

Project Dependency Diagram

Diagram Source

Opendaylight Beryllium Project Dependency Diagram.png

Communication Channels

Mailing List

The release mailing list (release@lists.opendaylight.org) is the formal channel for communication about the Simultaneous Release.

Please limit mail to this list to things that directly concern the release as our goal is to keep it's volume at a level that allows the project lead/contact to read all of it.

Per-project Simultaneous Release Contact

Each project participating in the Simultaneous Release should designate a committer to be the contact for that project for that Simultaneous Release. It is expected that this be the project lead for most projects. Even though a primary contact other than the project lead can be designated, the project lead is still expected to be ultimately responsible for the project's participation in the release.

Cross Project Milestone and Release Candidate Reporting

At each milestone, each project is expected to send a readout to the release mailing list by 23:59:59 UTC on the date listed for the given milestone and offset. Most information will be reported via the release tracking spreadsheet, which can be found in the supporting documents section. While most information will be reported via the spreadsheet, projects should still send a mail indicating the information has been filled in, reporting any extra information, and possibly asking additional questions. Reported information will include things like links to gerrit patches, pointers to continuous integration Jenkins Jobs, and the like.

Negative statuses should be reported promptly. If a project is under threat of, or does miss an element on its release plan, the project contact/lead should report this as soon as it is known. They should not wait until the next milestone's readout.

It is the responsibility of each project's lead to report both positive and negative statuses. While they can delegate the task, the project lead is still ultimately responsible for the project's participation in the release.

Simultaneous Release Developer Meetings

One week prior to each Milestone or Release Candidate starting at M1, an IRC meeting for developer interested in the Simultaneous Release will be organized for real time coordination and check in. The Project for each project (or their delegate) should minimally be in attendance. This meeting should happen for each offset at each milestone.

The meeting will be held in #opendaylight-meeting on freenode. You can use an IRC client of your choice or the freenode web client if it is easier.

Bugs

Bugzilla is used to track all bugs in OpenDaylight. Bugs must be filed for the appropriate project. General guidelines and sample searches can be found on the OpenDaylight Bugs page.

During the release candidate process, all blocking bugs must be both logged on a bug-tracking spreadsheet (to be provided) and filed appropriately, e.g., with severity set to BLOCKING, in Bugzilla.

Weather Page

To track current ongoing issues and upcoming possible issues, it's worth checking and updating the Weather page since it provides an easier to find location for ongoing things than the mailing list.

Cross Project Meetings

Supporting Documents

Milestone Readout Templates

Note that any deliverable missed in a previous milestone should be reported on in all subsequent milestone readouts until the deliverable is completed. Also note that additional questions may be added if we need to gather it from all projects, e.g., in Lithium we asked about issues with the migration to Karaf 3.0.3.

  • TODO: we need a place to track Karaf features with whether they are top-level and/or user-facing.

M1: Joining the Release

<Project Name>

  1. Offset xx
  2. Project Lead elected by committers: (name/email/IRC)
    • Be sure to read the responsibilities of being a project lead under Leadership & Communication in the Requirements for Participation section of the Beryllium release plan [0].
  3. Project Contact: (name/email/IRC)
  4. Test Contact: (name/email/IRC)
  5. Documentation Contact (name/email/IRC)
  6. Published draft release plan: (wiki link)
  7. Project Main page created using the Project Facts [1] template: (wiki link)
  8. A statement to the effect: "The <Project Name> project formally joins the OpenDaylight Beryllium Simultaneous Release and agrees to the activities and timeline documented on the Beryllium Release Plan Page: https://wiki.opendaylight.org/view/Simultaneous_Release:Beryllium_Release_Plan"

[0] https://wiki.opendaylight.org/view/Simultaneous_Release:Beryllium_Release_Plan#Requirements_for_Participation

[1] https://wiki.opendaylight.org/view/Template:Project_Facts

M2: Final Release Plan

<Project Name>

Please provide updates on any previously-incomplete items from prior milestone readouts.

  1. Have you completed the project checklist [0]? Yes/No
    • (link to a merged patch in gerrit)
    • (link to a mail from your mailing list)
    • (link to a bug for your project; you can create a dummy one and close it if need be)
    • (link to an artifact published from your project in nexus)
    • (link to a sonar report)
    • (link to your root pom file)
  2. Does your project have any special needs in CI Infrastructure (tools or configuration)? Yes/No
    • Note that generally, the only available tools in CI are basic RHEL/CentOS linux images with Java. You should note and ask for anything beyond that here.
    • If yes, have helpdesk [1] tickets been opened for these CI needs? Yes/No
  3. Is your project release plan finalized? Yes/No (wiki link)
    • If No, ETA to finalize release plan
    • Do you have all APIs intended to be externally consumable listed? Yes/No
      • Does each API have a useful short name?
      • Are the Java interface and/or YANG files listed for each API?
      • Are they labeled as tentative, provisional, or stable as appropriate for each API?
      • Do you call out the OSGi bundles and/or Karaf features providing the API for each API?
  4. All project dependencies (requests on other project's release plans) have been acknowledged and documented by upstream projects? Yes/No
  5. Were there any project-specific deliverables planned for this milestone? Yes/No
    • If so, were they delivered? Yes/No

Integration and Test:

  1. Will your project have top-level features not requiring system test? Yes/No
    • If yes, list them and explain why system test should be waived for them
  2. Will your project use the OpenDaylight CI infrastructure for testing top-level features requiring system test? Yes/No
    • If no, why? It is recommended to use the OpenDaylight CI infrastructure unless there is some HW or SW resource that cannot be installed there
    • if no, any plan for testing in external Lab? Projects running system test in external Labs are required to report system test results in a timely fashion after release creations, e.g., weekly, RC, and formal releases

[0] https://wiki.opendaylight.org/view/GettingStarted:Project_Main#New_Project_Checklist

[1] email helpdesk@opendaylight.org

M3: Functionality Freeze

<Project Name>

Please provide updates on any previously-incomplete items from prior milestone readouts.

Functionality Freeze:

  1. Final list of externally consumable APIs defined: Yes/No
    • If you had an Tentative APIs, have they been moved to Provisional or dropped? Yes/No (link to release plan)
    • If any of your Tentative APIs were dropped, have you notified all projects that were expecting them? Yes/No (link to e-mail)
      • Also please list all dropped APIs.
  2. Are all your inter-project dependencies are resolved (i.e., have the other projects you were counting on given you what you needed)? Yes/No
    • If no, please list the features you were expecting that haven't been delivered and the project you were expecting to receive them from.
    • Note that you can only reasonably hold a a project to something if you formally asked for it during the release planning process and they acknowledged that ask saying they would do it.
  3. Were there any project-specific deliverables planned for this milestone? Yes/No
    • If so, were they delivered? Yes/No

Karaf Features Defined:

  1. Are all your project's features that are intended for release added to the features.xml and checked into integration git repository. Yes/No (please provide link to the gerrit patch)
  2. List all top-level, user-facing, and stable Karaf features for your project.
    • For top-level and user-facing features, please provide a one-sentence description which a developer and/or user would find helpful.

Documentation:

  1. List the kinds of documentation you will provide including at least:
    • One user/operator guide section per user-facing feature.
    • One developer guide per top-level feature.
    • An installation guide for any top-level features that require more than feature:install <feature-name> to install.
    • Optional tutorials and how tos.
  2. Have you checked in an AsciiDoc outline to the docs repository? Yes/No (link to gerrit patch)

Integration and Test:

  1. Have you started automated system testing for your top-level features. Yes/No
    • If yes, link to test report
    • If no, why?
  2. Have you filled out basic system test plan template for each top-level feature (karaf and not karaf) and a comprehensive system test plan template including functionality, cluster, scalability, performance, longevity/stability for each stable feature? Yes/No
    • If yes, link to test plans
    • If no, why?

Project Specific:

  1. Were there any project-specific deliverables planned for this milestone? Yes/No
    • If so, were they delivered? Yes/No

M4: API Freeze

<Project Name>

  1. Please provide updates on any previously-incomplete items from prior milestone readouts.
  2. Has your project achieved API freeze, i.e., all externally accessible APIs [Stable and Provisional] will not be modified after now? Yes/No
    • Provide a link to a gerrit search for patches modifying the files defined as specifying the API. Example [1].
  3. What are the current word counts (total and per-document) for your project?
    • word count
    • links to gerrit patches or links to the file in git as appropriate
  4. Has your project met the requirements to be included in Maven Central [2]? Yes/No
  5. Were there any project-specific deliverables planned for this milestone? Yes/No
    • If so, were they delivered? Yes/No
  6. Have you started automated system testing for your top-level features. Yes/No
    • If yes, link to test report
    • If no, why?
  7. Have you updated the wiki [3] with any ports used, including for testing?
  8. Does your project build successful in Beryllium Autorelease [4]?
    • If not, explain why.
  9. Does your project build with Java 8?
    • Link to JDK8 verify job

[1] https://git.opendaylight.org/gerrit/#/q/file:%255Eopendaylight/md-sal/sal-binding-api/.%252B+status:merged+project:controller
[2] http://central.sonatype.org/pages/requirements.html
[3] https://wiki.opendaylight.org/view/Ports
[4] https://wiki.opendaylight.org/view/RelEng/Autorelease/Project_Autorelease_Requirements

M5: Code Freeze

<Project Name>

Please provide updates on any previously-incomplete items from prior milestone readouts.

  1. Has your project met code freeze, i.e., only bug fixes is allowed from now on? Yes/No
  2. Has your project cut the stability branch (stable/beryllium), pushed an update to the stable/beryllium .gitreview file pointing to stable/beryllium, and bumped version numbers on the master branch [0]? Yes/No
    • Please provide a link to the gerrit patch for the version bump on master
    • Please provide a link to the gerrit patch for the .gitreview file update on stable/beryllium
  3. Are all externally visible strings frozen to allow for translation & documentation? Yes/No
  4. Is your documentation complete, i.e., Only editing and enhancing should take place after this point? Yes/No (link to gerrit patch or merged documentation)
  5. Were there any project-specific deliverables planned for this milestone? Yes/No
    • If so, were they delivered? Yes/No

Integration and Test:

  1. Do your stable features fulfill quality requirements (i.e. unit and/or integration test coverage of at least 75%)? Yes/No
    • If yes, link to sonar report
    • If no, why?
  2. Are you running at least one basic automated system test job for each top-level feature? Yes/No
    • If yes, link to test report
    • If no, why?
  3. Are you running several automated system test jobs including functionality, cluster, scalability, performance, longevity/stability for each stable feature? Yes/No
    • If yes, link to test reports
    • If no, why?

[0] https://wiki.opendaylight.org/view/Simultaneous_Release:Cutting_Stability_Branches

RCX: Release Candidate Testing

<Project Name>

  1. Have you tested your code in the release candidate? Yes/No (provide a link to the release candidate you tested)
    • If yes, did you find any issues?
    • If you found issues, do you believe any of them should block this release of OpenDaylight until they are resolved?
    • Please list all the issues and note if they are blocking.

Lessons learned during the Beryllium release

  • Do we need enhanced requirements to be in a release and when to meet them?
  • If you're planning to develop major new functionality or replace the implementation of major current functionality, be cognizant of your downstream consumers
    • have a migration plan for how people will start using the new functionality (or implementation)
    • understand how to help them deal with bugs as they come up
    • if you will have two different simultaneous implementations, make they can both be run at the same time
      • or ideally, don't do this, make sure everyone can make the jump to the new implementation
  • upgradability is critical, there's no sane way to do this without
    1. a per-project requirement to be upgradable
    2. tests to confirm that this actually works
  • there have been requests to have upgrades work with hops between versions, e.g., Helium => Beryllium
  • we need stronger testing for upgradeablity between SRs
  • projects should have only stable tests. If there are failures, then:
    • mark the failure clearly with a known and acceptable bug
    • fix the bug
    • remove the test case if it is invalid
  • projects should be required to generate a maven site with javadoc, and ideally REST API doc generated from YANG models
    • ideally this should be done in a way that autorelease can generate one big maven site
  • projects don't do system test even when they say they do
  • We need to update release reviews and release notes to make us have to ask fewer questions
    • features (with links to the features.xml file and the right line for each feature to avoid listing non-existent features)
      • if there are any experimental features
      • user-facing features (with a clear definition of user-facing)
    • migration
    • compatibility (which is different from migration)
    • documentation (both wiki and AsciiDoc)
    • Ideally also add the wiki template to produce lists of release notes, release plans, and release reviews
  • It would be good to have documentation peer-review in a structured way
    • Maybe pairing up projects (or even forming small groups, e.g., 3 projects that have to do cross-review)

Lessons from Hydrogen, Helium and Lithium for future releases

Items that are struck out have been addressed. A comment will follow as to how it was addressed in blue if it seems like the right approach and in red if there may still be debate or questions. Similarly, comments in goldenrod note things that still need to be addressed, but are not blocking certifying the release plan.

  • Do we want to require system tests for all top-level features (and thus one per project) instead of just user-facing features? done, it's for top-level features
  • The release notes and release review templates need to be revised
    • Likely combined and to use a wiki template to make them more uniform and easier to suck out For now, the release notes have been updated to be more clearly per-project. The release review template is specified by the lifecycle document and so has been left the same and separate.
  • We need to track items that weren't green from previous milestone readouts so that they get reported in subsequent ones until they are green
    • A key example is getting features into integration
  • We need to require project feedback on RCs as part of the release plan.
  • Service releases should likely continue until some future release (either one or two releases in the future) rather than after a fixed number of releases. we've moved to four SRs with the note that the should go for at least one release in the future
    • How long after the new release do we wait?
    • Do we want to have a specified amount of overlap? 2 weeks? 6 weeks?
  • We desperately need pre-made templates for each milestone that make verifying requirements easy. We just missed things in Lithium M1 to M3 without that.
  • We need to mandate source jars generated in a canonical way, i.e., to be consumed by releng/autorelease.
  • We should also mandate javadoc generated in a canonical way, i.e., to be consumed by releng/autorelease. Thanh says releng is already doing this both in JJB for projects and auto-release.
  • We need to mandate that no projects can pull in 3rd-party dependencies without source jars.
  • Migration migration and backward compatibility requirements have been put out as part of mature projects and/or stable features, but could maybe use to be made more concrete
    • Do we want to require data schema translations?
    • Other issues?
  • The paperwork for M3 was substantial (and not easy see in advance) and should be streamlined or spread out to dos have been added above to copy documentation and integration requirements into this document to keep them from sneaking up on people
    • Consolidating all the requirements into one place would likely help.
  • We should make sure that people know where to produce and document known issues. In general, it's three places:
    • The release mailing list.
    • The Weather page.
    • The weekly IRC sync during the last part of the release. This is called out in #Communication Channels and any more forceful call out will likely have to happen not in this document
  • Make it clear what is expected of projects in terms of tracking what's going on in ODL.
    • Reading the release list.
    • Reading tsc list or at least the TSC meeting minutes.
    • Attending release IRC meetings or at least reading the minutes. This is called out in the Communication & Leadership requirement above and that is pointed out in the M1 milestone readout template
  • Adding a way to deal with docs-like projects that don't provide code-level negative interactions
    • This includes at least docs, toolkit (now mostly defunct), and coretutorials
    • Ideally, they might have laxer requirements for now, I think leaving these projects as leaf projects with minimal requirements seems reasonable
  • Deal with cutting branches and version bumps with offsets TODO: for now we've left the exact timing of branch cutting open to sometime between offset 0 M5 and offset 2 M5 as well as leaving open whether we'll do a synchronized cut or a cut staggered by offset.
    • If we cut branches and version bump at the same time, the only issue we have is slow projects that can hold things up, which has been fixed by automated version bumping that should happen for Helium.
    • However, if we cut branches and version bump at M5, there is an offset between different projects where upstream projects can merge patches that break downstream projects (regardless of whether the breakage is a bug in the upstream or downstream project) causing the version bump for downstream projects to fail.
    • The options seem to be:
      1. cut branches all at the same time at or near M5-offset2 or RC0, which has the disadvantage that offset 0 and offset 1 projects have somewhere between a 2-week and 6-week period after code freeze where they can't add new features.
      2. cut branches all at the same time at M5-offset0, which has the disadvantage that offset 1 and offset 2 projects will have to "double merge" patches to both branches during times that would normally have heavy coding.
      3. figure out how to deal with the issue that offset 1 and offset 2 projects may get hit with incompatibilities on version bumps
    • Also, cascading tests during staggered branch cutting breaks because the way JJB is set up right now it's not possible to trigger jobs across branches even when logically master of an offset 1 project is dependent on stable/lithium of an offset 0 project.
  • Docs improvements
    • We'd really like to be able to put project-specific docs in their repo
      • This will allow us to easier iterate on docs as patches go in, and allow committers to +2/-2 doc changes. (alagalah)
      • This should include the ability to directly pull code fragments from real code
    • We'd really like HTML versions of the docs that aren't so fragmented This will be folded in to the docs project's Beryllium release plan

Lessons from Hydrogen/Helium that Should be Applied

Items that are struck out have been addressed. A comment will follow as to how it was addressed in blue if it seems like the right approach and in red if there may still be debate or questions. Similarly, comments in goldenrod note things that still need to be addressed, but are not blocking certifying the release plan.

  • The Release plan doesn't take into account project dependencies. e.g. M4 API Freeze. If a project is waiting on API freeze for a project it is dependent on, then that reduces the amount of time the "dependee" has to execute. - alagalah (Keith) Done, mainly by moving deadlines up by one step, e.g., M2 component free, M3 API freeze, M4 code freeze
    • We had offsets in Hydrogen, spaced at 2 days. We need 2-3 weeks between offsets for them to make sense,
    • With 6 offsets 2 weeks each we need additional 10 weeks to reach RC0 on all projects,
      • Can we can do it in 3 offsets: +0, +2 weeks, +4 weeks
        1. odlparent, yangtools, and controller
        2. openflowjava, openflowplugin, ovsdb, aaa
        3. everyone else
    • Which means lower-offset projects can (and need) to start their next-release while the SR process is finishing
  • We need a Feature Freeze milestone before the API freeze
    • It should occur at M3 with beta-quality APIs, so downstream projects can start consuming Currently at M2 instead, it will be ~M2.5 and ~M3 for most projects
  • We're using release@lists.opendaylight.org instead of discuss
  • We should make it easy for projects to convey and understand what APIs they are intending to make available vs. which ones are intended to be internal attempted as part of component/API freeze
  • We should make it clear that participation in Service Releases is not optional done, see #Requirements for Participation
  • We should make it clear what we expect in terms of timely responses from project primary contacts for a release done, see #Requirements for Participation
    • This involves identifying what mails that people should pay attention to, e.g., ones sent to release@lists.opendaylight.org with "PLEASE RESPOND" in the subject
    • It also involves identifying a time frame in which they should respond, e.g., two business days
      • One concrete stab at making this formal would be: "Technically, two business days will be defined as 48 hours not counting 2a UTC on Friday until 2a UTC on Sunday. This corresponds to 48 hours starting at 4p on Friday in the furthest ahead time zone (UTC+14). Note that this means if you want a response *this* week, you must send it before 2a UTC on Wednesday. That’s 6/7p pacific time on Tuesday in the Pacific time zone." The formal definition is currently left out
  • We need a longer time between code freeze and release candidates because developers don't focus on tests (especially system and integration tests) until after code freeze
  • Status reports for each milestone should include more than a Boolean for tests
  • We need to make it clear what tasks need to be done for docs, where and when handled by the deliverable from M2 from docs
    • Understanding the kinds of documentation we want to generate and who the audience is for each kind is going to be critical
      • e.g., one person's user is likely another's developer
    • The same is true about tests. handled by the deliverable from M2 from integration
  • We really need somebody who groks the things that need to be accomplished at each milestone and can take a glance at the code and jenkins jobs for each project to get an idea of whether they're on track or not. We need to make sure we do this for deadlines M3 and later, e.g., functionality freeze, karaf features defined, API freeze, and code freezeM milestone readout templates make this more doable
  • Requirements to meet at different stages (and especially RCs) should be set and enforced with clearly explained consequences for missing them ways to fix missed deadlines are now discussed by the TSC for offset zero and offset one projects as described in requirements for participation
    • Release throttle branch needs to be cut at RC0 at the latest done at M5 now
  • We need a standard way to track blocking issues: TODO: we still need this, but it's loosely defined in the Helium blocking bugs section here.
    • One suggestion is to treat them as bugs in bugzilla for easy tracking and querying
      • Projects would file bugs with severity as "critical", "blocker" with the target milestone being appropriate
      • Appropriate milestones are sometimes annoying, but generally, it should be "anything but the next release"
  • We need to pre-declare when RCs and final release artifacts will be cut (both dates and times for clarity) done at M5 and RCs
  • Need to add an EOL-plans section to release plan to understand user impact of EOLed features/components/APIs at the start of a development cycle done in release plan
    • What requirements do we want to place on projects? e.g., deprecated in one release and can remove in another?
    • plans for dealing with EOLed features should be incorporated into the release plan
  • We should reconsider when we set a release date done, there is a month of slack between the release and the ODL summit and the dates for RCs after RC0 and the formal release are stated to be tentative based on testing in the definition of RCs
    • Especially to the press, but also in other environments
    • For example, do we want to have a booked event giving us effectively zero wiggle room on the back end?
      • Maybe, because hard deadlines help get things done, but they also make for sub-optimal
  • We could use more automated release processes
    • For example, the auto-release is really, really nice as compared to spending 14+ hours on IRC cutting everything.
    • A similar process for post-release branch cutting and version bumping would be very helpful, e.g., take a 10+ day process and turn it into one that takes a few hours.
    • One problem is figuring out how to do this w/o requiring involvement from every project (at least on the critical path).
      • Solutions are (i) allowing for some scripts to commit changes to projects, which is likely bad, or (ii) automatically pushing patches for projects to review
      • Another solution is to switch to continuous delivery resolved thanks to Thanh and the autorelease project
  • We should avoid scheduling any major events, e.g., a design forum or summit, immediately after the release so that we can have some room for slippage without having to pull many developers out of the event into a "war" room. done. there is a month between the release and the ODL summit
  • More automated features testing TODO: yes, but this is technical debt, not directly related to the release plan
    • to really test things, we need to blow away the m2 repo before testing every features.xml file
  • Cyclic dependencies TODO: yes, but this is technical debt, not directly related to the release plan
    • We need to decide if we want to allow them, and if so what kind to allow
    • We need to provide documentation (or ideally scripts) that show how to build the code despite the circular dependencies (if we allow them)
    • We need tests to check for circular dependencies (either at all or new ones) so that we know about them
      • The simplest way to do this would be to have an offline auto-release which first clone all the repos and then tried to build them linearly without access to the nexus repos.