Skip to content

[action] [PR:17889] Disable PFC-WD during PCBB and some wmk test improvements#17991

Merged
mssonicbld merged 1 commit intosonic-net:202411from
mssonicbld:cherry/202411/17889
Apr 16, 2025
Merged

[action] [PR:17889] Disable PFC-WD during PCBB and some wmk test improvements#17991
mssonicbld merged 1 commit intosonic-net:202411from
mssonicbld:cherry/202411/17889

Conversation

@mssonicbld
Copy link
Copy Markdown
Collaborator

Description of PR

Summary:
During flaky test analysis of:

  • qos/test_tunnel_qos_remap.py::test_pfc_watermark_extra_lossless_active
  • qos/test_tunnel_qos_remap.py::test_pfc_watermark_extra_lossless_standby

Discovered that PFC-WD was enabled and being triggered during these tests. Added an auto-use fixture to disable PFC-WD and restore appropriately, similar to QOS SAI fixture.
Included the packet aging logic in case a relevant asic needs it going forward.

Changes:

  1. Disable PFC-WD during all PCBB tests
  2. Add a sleep after PFC-storm stop in PCBB tests to ensure packets are able to exit before further checks are performed.
  3. Improve failure logging for above 2 tests.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • New Test case
  • Skipped for non-supported platforms
  • Test case improvement

Back port request

  • 202012
  • 202205
  • 202305
  • 202311
  • 202405
  • 202411

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Validated: 1dfc735

  • Both _active and _standby watermark tests pass on Cisco-8000 (3X executions)

Sample failure for new logging code:

E AssertionError: Watermark failures were found:
E For inner_dscp, outer_dscp, prio, queue = (3, 2, 3, 3):
E Failed to detect congestion due to PFC pause, failed check 0 > 10000

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

…17889)

* Improve validation so multiple failures can be reported.

* Disable PFC-WD during PCBB tests.

* Wait for PFC-WD to stop.

* Remove packet aging code from pfcwd fixture.
@mssonicbld
Copy link
Copy Markdown
Collaborator Author

Original PR: #17889

@mssonicbld
Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@wsycqyz
Copy link
Copy Markdown
Contributor

wsycqyz commented Apr 16, 2025

Rerun failed jobs.

@mssonicbld mssonicbld merged commit d5bd995 into sonic-net:202411 Apr 16, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants