Update quality declaration and coverage (#674)

* Update quality level and coverage

Signed-off-by: ahcorde <ahcorde@gmail.com>

* Added feedback

Signed-off-by: ahcorde <ahcorde@gmail.com>

* Fixed wording and links

Signed-off-by: ahcorde <ahcorde@gmail.com>

* Updated QD

Signed-off-by: ahcorde <ahcorde@gmail.com>

Author: ahcorde

rcl/QUALITY_DECLARATION.md

@@ -2,7 +2,7 @@ This document is a declaration of software quality for the `rcl` package, based
 
 # `rcl` Quality Declaration
 
-The package `rcl` claims to be in the **Quality Level 4** category.
+The package `rcl` claims to be in the **Quality Level 3** category.
 
 Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Quality Categories in REP-2004](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-quality-categories).
 
@@ -101,7 +101,9 @@ Currently nightly test results can be seen here:
 
 ### Public API Testing [4.ii]
 
-Most of the functionality of the declared API in this package is covered in its unit tests. Currently it has a line coverage of [77%](https://ci.ros2.org/job/ci_linux_coverage/85/cobertura/src_ros2_rcl_rcl_src_rcl/).
+Each part of the public API has tests, and new additions or changes to the public API require tests before being added. The tests aim to cover both typical usage and corner cases, but are quantified by contributing to code coverage.
+
+The following functions are partially supported: `rcl_take_loaned_message`, `rcl_return_loaned_message_from_subscription`, `rcl_borrow_loaned_message`, `rcl_return_loaned_message_from_publisher` and `rcl_publish_loaned_message` because they are not currently supported on Tier 1 RMW providers.
 
 ### Coverage [4.iii]
 
@@ -114,7 +116,7 @@ This includes:
 
 Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers.
 
-Current coverage statistics can be viewed (77%) [here](https://ci.ros2.org/job/ci_linux_coverage/85/cobertura/src_ros2_rcl_rcl_src_rcl/). Current coverage statistics need to be improved to reach a higher quality level.
+Current coverage statistics can be viewed [here](https://ci.ros2.org/job/nightly_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rcl_rcl_src_rcl/). A description of how coverage statistics are calculated is summarized in this page ["ROS 2 Onboarding Guide"](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).
 
 ### Performance [4.iv]
 
@@ -167,7 +169,7 @@ It is **Quality Level 4**, see its [Quality Declaration document](https://github
 
 The `rcutils` package provides an API which contains common utilities and data structures needed when programming in C.
 
-It is **Quality Level 4**, see its [Quality Declaration document](https://github.com/ros2/rcutils/blob/master/QUALITY_DECLARATION.md).
+It is **Quality Level 2**, see its [Quality Declaration document](https://github.com/ros2/rcutils/blob/master/QUALITY_DECLARATION.md).
 
 #### `rmw_implementation`
 

rcl/README.md

@@ -4,4 +4,4 @@ Library to support implementation of language specific ROS Client Libraries.
 
 Features are described in detail at [http://docs.ros2.org](http://docs.ros2.org/latest/api/rcl/index.html)
 
-This package claims to be in the **Quality Level 4** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.
+This package claims to be in the **Quality Level 3** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.

rcl_action/QUALITY_DECLARATION.md

@@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rcl_action` package,
 
 # `rcl_action` Quality Declaration
 
-The package `rcl_action` claims to be in the **Quality Level 4** category.
+The package `rcl_action` claims to be in the **Quality Level 3** category.
 
-Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 4 in REP-2004](https://www.ros.org/reps/rep-2004.html).
+Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Quality Categories in REP-2004](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-quality-categories).
 
 ## Version Policy [1]
 
@@ -88,15 +88,29 @@ The results of the test can be found [here](https://ci.ros2.org/view/nightly/job
 
 `rcl_action` has feature tests, which test for proper node state transitions.
 The tests are located in the [test](test) subdirectory.
+New features are required to have tests before being added.
+Currently nightly test results can be seen here:
+* [linux-aarch64_release](https://ci.ros2.org/view/nightly/job/nightly_linux-aarch64_release/lastBuild/testReport/rcl_action/)
+* [linux_release](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rcl_action/)
+* [mac_osx_release](https://ci.ros2.org/view/nightly/job/nightly_osx_release/lastBuild/testReport/rcl_action/)
+* [windows_release](https://ci.ros2.org/view/nightly/job/nightly_win_rel/lastBuild/testReport/rcl_action/)
 
 ### Public API Testing [4.ii]
 
-Much of the API in `rcl_action` is tested.
-The tests are located in the [test](test) subdirectory.
+Each part of the public API has tests, and new additions or changes to the public API require tests before being added. The tests aim to cover both typical usage and corner cases, but are quantified by contributing to code coverage.
 
 ### Coverage [4.iii]
 
-`rcl_action` does not currently track test coverage.
+`rcl_action` follows the recommendations for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#code-coverage), and opts to use line coverage instead of branch coverage.
+
+This includes:
+
+- tracking and reporting line coverage statistics
+- no lines are manually skipped in coverage calculations
+
+Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers.
+
+Current coverage statistics can be viewed [here](https://ci.ros2.org/job/nightly_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rcl_rcl_action_src_rcl_action/). A description of how coverage statistics are calculated is summarized in this page ["ROS 2 Onboarding Guide"](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).
 
 ### Performance [4.iv]
 
@@ -129,13 +143,13 @@ It is **Quality Level 4**, see its [Quality Declaration document](https://github
 
 `rcl` is the ROS 2 client library in C.
 
-It is **Quality Level 4**, see its [Quality Declaration document](../rcl/QUALITY_DECLARATION).
+It is **Quality Level 3**, see its [Quality Declaration document](../rcl/QUALITY_DECLARATION).
 
 #### `rcutils`
 
 `rcutils` provides commonly used functionality in C.
 
-It is **Quality Level 4**, see its [Quality Declaration document](https://github.com/ros2/rcutils/blob/master/QUALITY_DECLARATION.md).
+It is **Quality Level 2**, see its [Quality Declaration document](https://github.com/ros2/rcutils/blob/master/QUALITY_DECLARATION.md).
 
 #### `rmw`
 

rcl_action/README.md

@@ -9,4 +9,4 @@ http://design.ros2.org/articles/actions.html
 
 ## Quality Declaration
 
-This package claims to be in the **Quality Level 4** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.
+This package claims to be in the **Quality Level 3** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.

rcl_lifecycle/QUALITY_DECLARATION.md

@@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rcl_lifecycle` packa
 
 # `rcl_lifecycle` Quality Declaration
 
-The package `rcl_lifecycle` claims to be in the **Quality Level 4** category.
+The package `rcl_lifecycle` claims to be in the **Quality Level 3** category.
 
-Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 4 in REP-2004](https://www.ros.org/reps/rep-2004.html).
+Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Quality Categories in REP-2004](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-quality-categories).
 
 ## Version Policy [1]
 
@@ -88,14 +88,30 @@ The results of the test can be found [here](https://ci.ros2.org/view/nightly/job
 
 `rcl_lifecycle` has feature tests, which test for proper node state transitions.
 The tests are located in the [test](test) subdirectory.
+New features are required to have tests before being added.
+Currently nightly test results can be seen here:
+* [linux-aarch64_release](https://ci.ros2.org/view/nightly/job/nightly_linux-aarch64_release/lastBuild/testReport/rcl_lifecycle/)
+* [linux_release](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rcl_lifecycle/)
+* [mac_osx_release](https://ci.ros2.org/view/nightly/job/nightly_osx_release/lastBuild/testReport/rcl_lifecycle/)
+* [windows_release](https://ci.ros2.org/view/nightly/job/nightly_win_rel/lastBuild/testReport/rcl_lifecycle/)
+
 
 ### Public API Testing [4.ii]
 
-Much of the API in `rcl_lifecycle` is tested in the aforementioned feature tests, but it is not tested explicitly.
+Each part of the public API has tests, and new additions or changes to the public API require tests before being added. The tests aim to cover both typical usage and corner cases, but are quantified by contributing to code coverage.
 
 ### Coverage [4.iii]
 
-`rcl_lifecycle` does not currently track test coverage.
+`rcl_lifecycle` follows the recommendations for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#code-coverage), and opts to use line coverage instead of branch coverage.
+
+This includes:
+
+- tracking and reporting line coverage statistics
+- no lines are manually skipped in coverage calculations
+
+Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers.
+
+Current coverage statistics can be viewed [here](https://ci.ros2.org/job/nightly_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rcl_rcl_lifecycle_src/). A description of how coverage statistics are calculated is summarized in this page ["ROS 2 Onboarding Guide"](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).
 
 ### Performance [4.iv]
 
@@ -128,7 +144,7 @@ It is **Quality Level 4**, see its [Quality Declaration document](https://github
 
 `rcl` is the ROS 2 client library in C.
 
-It is **Quality Level 4**, see its [Quality Declaration document](../rcl/QUALITY_DECLARATION).
+It is **Quality Level 3**, see its [Quality Declaration document](../rcl/QUALITY_DECLARATION).
 
 #### `rcutils`
 

rcl_lifecycle/README.md

@@ -9,4 +9,4 @@ https://design.ros2.org/articles/node_lifecycle.html
 
 ## Quality Declaration
 
-This package claims to be in the **Quality Level 4** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.
+This package claims to be in the **Quality Level 3** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.

rcl_yaml_param_parser/QUALITY_DECLARATION.md

@@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rcl_yaml_param_parse
 
 # `rcl_yaml_param_parser` Quality Declaration
 
-The package `rcl_yaml_param_parser` claims to be in the **Quality Level 4** category.
+The package `rcl_yaml_param_parser` claims to be in the **Quality Level 3** category.
 
-Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 4 in REP-2004](https://www.ros.org/reps/rep-2004.html).
+Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Quality Categories in REP-2004](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#package-quality-categories).
 
 ## Version Policy [1]
 
@@ -59,10 +59,12 @@ All pull requests must resolve related documentation changes before merging.
 
 ### Feature Documentation [3.i]
 
-`rcl_yaml_param_parser` does not have feature documentation.
+`rcl_yaml_param_parser` provides the main elements of its API listed using doxygen and is hosted [here](http://docs.ros2.org/latest/api/rcl_yaml_param_parser/index.html).
 
 ### Public API Documentation [3.ii]
 
+`rcl_yaml_param_parser` has embedded API documentation and it is generated using doxygen. Currently, its latest version is hosted [here](http://docs.ros2.org/latest/api/rcl_yaml_param_parser/index.html). Latest version has to be generated before considering this item fully resolved.
+
 All of `rcl_yaml_param_parser` has embedded API documentation. It is not yet hosted publicly.
 
 ### License [3.iii]
@@ -85,15 +87,30 @@ The results of the test can be found [here](https://ci.ros2.org/view/nightly/job
 
 ### Feature Testing [4.i]
 
-`rcl_yaml_param_parser` has feature tests, which ensure that a wide variety of yaml files are checked for parsing.
+Most features in `rcl_yaml_param_parser` have corresponding tests which simulate typical usage, and they are located in the [`test`](./test) directory.
+New features are required to have tests before being added.
+Currently nightly test results can be seen here:
+* [linux-aarch64_release](https://ci.ros2.org/view/nightly/job/nightly_linux-aarch64_release/lastBuild/testReport/rcl_yaml_param_parser/)
+* [linux_release](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rcl_yaml_param_parser/)
+* [mac_osx_release](https://ci.ros2.org/view/nightly/job/nightly_osx_release/lastBuild/testReport/rcl_yaml_param_parser/)
+* [windows_release](https://ci.ros2.org/view/nightly/job/nightly_win_rel/lastBuild/testReport/rcl_yaml_param_parser/)
 
 ### Public API Testing [4.ii]
 
-Much of the API in `rcl_yaml_param_parser` is tested in the aforementioned feature tests, but it is not tested explicitly.
+Each part of the public API has tests, and new additions or changes to the public API require tests before being added. The tests aim to cover both typical usage and corner cases, but are quantified by contributing to code coverage.
 
 ### Coverage [4.iii]
 
-`rcl_yaml_param_parser` does not currently track test coverage.
+`rcl_yaml_param_parser` follows the recommendations for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#code-coverage), and opts to use line coverage instead of branch coverage.
+
+This includes:
+
+- tracking and reporting line coverage statistics
+- no lines are manually skipped in coverage calculations
+
+Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers.
+
+Current coverage statistics can be viewed [here](https://ci.ros2.org/job/nightly_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rcl_rcl_yaml_param_parser_src/). A description of how coverage statistics are calculated is summarized in this page ["ROS 2 Onboarding Guide"](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs).
 
 ### Performance [4.iv]
 
@@ -118,7 +135,7 @@ It also has several test dependencies, which do not affect the resulting quality
 
 `rcutils` provides commonly used functionality in C.
 
-It is **Quality Level 4**, see its [Quality Declaration document](https://github.com/ros2/rcutils/blob/master/QUALITY_DECLARATION.md).
+It is **Quality Level 3**, see its [Quality Declaration document](https://github.com/ros2/rcutils/blob/master/QUALITY_DECLARATION.md).
 
 #### `libyaml_vendor`
 

rcl_yaml_param_parser/README.md

@@ -30,4 +30,4 @@ This package depends on C libyaml.
 
 ## Quality Declaration
 
-This package claims to be in the **Quality Level 4** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.
+This package claims to be in the **Quality Level 3** category, see the [Quality Declaration](./QUALITY_DECLARATION.md) for more details.