Markdown format for compare operation result

[enqueue]

We are using dbt-coverage for checking that PRs won't lead to worse test coverage, i.e. by comparing the report generated in the PR branch against the report that was created for the branch the PR is targeting (usually "main"). It would be nice if we could feed the output of the comparison tool into a github PR or a check annotation as Markdown content.

[vvvito]

You can do this natively using $GITHUB_STEP_SUMMARY in a github action step

      # Compare the documentation coverage between this branch and master.
      - name: Docs Coverage Comparison
        run: |
          
          # Now compare this PRs doc coverage to prod
          echo "# Coverage Comparison" >> $GITHUB_STEP_SUMMARY
          echo "~~~" >> $GITHUB_STEP_SUMMARY
          echo "$(dbt-coverage compare \coverage.json \prod_coverage.json)" >> $GITHUB_STEP_SUMMARY
          echo "~~~" >> $GITHUB_STEP_SUMMARY
          echo "** before = _prod_, after = _this branch_ **" >> $GITHUB_STEP_SUMMARY

We have an action step that will perform the comparison of the two coverage reports (prod one is from the main branch) then output to the Github Action Summary, resulting in the below:

[enqueue]

Thanks for the explanation. I managed to set this up by comparing against artifact stored for main workflow run. In addition to Action Summary we are feeding the output into a comment.

However, it takes some lines of awk magic to bring the result into beautiful MarkDown format, so it would be nice to have an option to output the comparison directly as MarkDown, as an alternative to the current ASCII art format.

[vvvito]

Ah gotcha - I was a bit bored so I think i managed to build what you're after into the code
i.e.

dbt_coverage compare --comp-format markdown  coverage.json prod_coverage.json

will print out

|           |   before      |      after      |  +/- |
|:----------|:-------------:|:---------------:|:----:|
| Coverage  |     34.97% |     32.87% |     -2.10% |
| Tables    |         13    |         13    | +0/+0    |
| Columns   |        286   |        286   | +0/+0   |
| Hits      |        100      |         94      | +0/-6      |
| Misses    |        186    |        192    | +6/+0    |

| Catalog | 100/286 (34.97%) | -> | 94/286 (32.87%) |
|:----|:----:|:--:|----:|
| ┌ `dim.dim_core_patients` | 25/39 (64.10%) | -> | 22/39 (56.41%) |
| ├── `date_of_birth_estimation_flag` | 1/1 (100.00%) | -> | 0/1 (0.00%) |
| ├── `family_name_preferred` | 1/1 (100.00%) | -> | 0/1 (0.00%) |
| └── `mrn` | 1/1 (100.00%) | -> | 0/1 (0.00%) |
| ┌ `dim.dim_core_orders` | 21/22 (95.45%) | -> | 20/22 (90.91%) |
| └── `name_cvid` | 1/1 (100.00%) | -> | 0/1 (0.00%) |
| ┌ `dim.dim_core_patient_current_identifiers` | 8/11 (72.73%) | -> | 6/11 (54.55%) |
| ├── `person_id` | 1/1 (100.00%) | -> | 0/1 (0.00%) |
| └── `tac number` | 1/1 (100.00%) | -> | 0/1 (0.00%) |

which will format out like this:

	before	after	+/-
Coverage	34.97%	32.87%	-2.10%
Tables	13	13	+0/+0
Columns	286	286	+0/+0
Hits	100	94	+0/-6
Misses	186	192	+6/+0

Catalog	100/286 (34.97%)	->	94/286 (32.87%)
┌ dim.dim_core_patients	25/39 (64.10%)	->	22/39 (56.41%)
├── date_of_birth_estimation_flag	1/1 (100.00%)	->	0/1 (0.00%)
├── family_name_preferred	1/1 (100.00%)	->	0/1 (0.00%)
└── mrn	1/1 (100.00%)	->	0/1 (0.00%)
┌ dim.dim_core_orders	21/22 (95.45%)	->	20/22 (90.91%)
└── name_cvid	1/1 (100.00%)	->	0/1 (0.00%)
┌ dim.dim_core_patient_current_identifiers	8/11 (72.73%)	->	6/11 (54.55%)
├── person_id	1/1 (100.00%)	->	0/1 (0.00%)
└── tac number	1/1 (100.00%)	->	0/1 (0.00%)

I can probably raise a PR for this if this is something you're after, or if there's anything else to add?

[enqueue]

Wow, that would make my life easier 😄 Just a cosmetic thing: I think the column headers in the lower table could be improved, so you won't need the column with the -> characters. Instead use "Before" and "After" just as in the upper table.

[vvvito]

Yeah - that makes more sense

New output would be:

	before	after	+/-
Coverage	34.97%	32.87%	-2.10%
Tables	13	13	+0/+0
Columns	286	286	+0/+0
Hits	100	94	+0/-6
Misses	186	192	+6/+0

	before (%)	after (%)
Catalog	100/286 (34.97%)	94/286 (32.87%)
┌ dim.dim_core_patients	25/39 (64.10%)	22/39 (56.41%)
├── mrn	1/1 (100.00%)	0/1 (0.00%)
├── date_of_birth_estimation_flag	1/1 (100.00%)	0/1 (0.00%)
└── family_name_preferred	1/1 (100.00%)	0/1 (0.00%)
┌ dim.dim_core_orders	21/22 (95.45%)	20/22 (90.91%)
└── name_cvid	1/1 (100.00%)	0/1 (0.00%)
┌ dim.dim_core_patient_current_identifiers	8/11 (72.73%)	6/11 (54.55%)
├── tac number	1/1 (100.00%)	0/1 (0.00%)
└── person_id	1/1 (100.00%)	0/1 (0.00%)

[joshuataylor]

I know this is a random fly-by comment, but I'm a huge fan of the output of the table.

I've been fiddling with trying to visualise tables/columns for a related project, and that's a really novel approach, well done. 👍