docs: partial reports can be submitted

[mlavacca]

What type of PR is this?

/kind documentation
/area conformance

What this PR does / why we need it:

After discussing the possibility of accepting partial reports in #3021, we eventually decided to encourage implementations to start submitting reports even if there is no full conformance. This PR updates the documentation to reflect this final decision.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

[mlavacca]

/cc @robscott @youngnick @shaneutt

[shaneutt]

After thinking it over, I do agree that it's better to get partial reports rather than no reports because this will increase our awareness of implementations in the field and enable more opportunities to engage with them.

I do think we may need to adjust the display to accommodate (make sure it's clear, and perhaps delineated when an implementation is reporting partial). I approve,

/approve

However I would like us to make sure we have a follow-up issue in place for considering how the display should look like for this (I don't consider it a blocker though) before we merge:

/hold

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mlavacca, shaneutt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• conformance/OWNERS [mlavacca,shaneutt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mlavacca]

After thinking it over, I do agree that it's better to get partial reports rather than no reports because this will increase our awareness of implementations in the field and enable more opportunities to engage with them.

I do think we may need to adjust the display to accommodate (make sure it's clear, and perhaps delineated when an implementation is reporting partial). I approve,

/approve

However I would like us to make sure we have a follow-up issue in place for considering how the display should look like for this (I don't consider it a blocker though) before we merge:

/hold

Yes, it makes sense. I've opened #3183 to capture this. Let me know if it fits with what you had in mind :)

[shaneutt]

LGTM

/unhold
/cc @robscott @youngnick

[robscott]

Thanks @mlavacca! LGTM, just a couple nits where further clarification could be useful.

[robscott]

It seems like we probably need to include some guidelines for how we handle partial conformance results. Maybe it's sufficient to just say that they'll be displayed separately?

[robscott]

I think there's another reason for partial conformance - the tests may all pass, but the reproduction required some kind of additional unexpected steps.

[mlavacca]

👍 I reworded this section to include this information

[youngnick]

All of @robscott's feedback is addressed, going to get this one in.

/lgtm