Azure DevOps: ReportGenerator vs PublishCodeCoverageResults

[bhuwan-agarwal]

I'm trying to understand the difference between the code coverage reports generated via the PublishCodeCoverageResults vs ReportGenerator Azure pipeline extension tasks. From the initial impressions they look quite the same.

Any details will be helpful?

[danielpalme]

As long as you want the coverage results of a single Cobertura file and you don't want to use any of the parameters of the ReportGenerator Azure pipeline extension task, you probably won't notice any difference.

If you want to use the latest version of ReportGenerator or if you want to to use specify any parameters, you should use the ReportGenerator Azure pipeline extension task.

The main issue with the PublishCodeCoverageResults@1 task is, that it regenerates the HTML report based on the supplied Cobertura file and ignores the HTML files generated by ReportGenerator.
This may result in unexpected behavior. See:

• Publish code coverage task overrides ReportGenerator filters microsoft/azure-pipelines-tasks#10354
• Mismatch between Branch Coverage in Cobertura and HTML reports #393
• Seems filters don't work #240

You can prevent the report regeneration with the following environment variable:

disable.coverage.autogenerate: 'true'

The environment variable is not well documented and the default behavior is not what users may expect.

How to decide

Tool	Advantages	Disadvantages
ReportGenerator Azure pipeline extension task	- Latest version of ReportGenerator - Support for parameters (e.g. filters) - Support for merging several coverage files - Ability to publish coverage results with setting publishCodeCoverageResults: true	- No diff coverage for PRs
PublishCodeCoverageResults@1	- Easy to setup for a single coverage file	- Deprecated - Regenerates HTML report with outdated version of ReportGenerator (can be disabled) - No support for parameters (e.g. filters) - No support for merging several coverage files
PublishCodeCoverageResults@2	- Easy to setup for a single coverage file - Diff coverage for PRs	- Regenerates HTML report with outdated version of ReportGenerator (can not be disabled) - No support for parameters (e.g. filters) - No support for merging several coverage files

Additional information can be found in the wiki.

[cremor]

Another advantage of the PublishCodeCoverageResults@2 task is that it allows the pull request page to show line coverage indicators for added/changed lines. (Green checkmarks show covered lines, red crosses show uncovered lines.)

Documentation

Example screenshots: