T2:snappi-tests:Add deviation modification for cisco-8000.#19598
Merged
yejianquan merged 1 commit intosonic-net:masterfrom Jul 22, 2025
Merged
T2:snappi-tests:Add deviation modification for cisco-8000.#19598yejianquan merged 1 commit intosonic-net:masterfrom
yejianquan merged 1 commit intosonic-net:masterfrom
Conversation
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
zhixzhu
approved these changes
Jul 14, 2025
Contributor
Author
|
/azpw run |
Collaborator
|
/AzurePipelines run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
auspham
approved these changes
Jul 16, 2025
Collaborator
|
Offline synced with @abdosi , it's lua script in syncd, might not able to modify the priority, we're good to modify the deviation for now. |
mssonicbld
pushed a commit
to mssonicbld/sonic-mgmt
that referenced
this pull request
Jul 22, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
11 tasks
Collaborator
|
Cherry-pick PR to 202505: #19760 |
11 tasks
Collaborator
|
Cherry-pick PR to msft-202405: Azure/sonic-mgmt.msft#567 |
mssonicbld
added
Included in msft-202405 Branch
and removed
Created PR to msft-202405 Branch
labels
mssonicbld
pushed a commit
that referenced
this pull request
Jul 22, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
mssonicbld
added
Included in 202505 branch
and removed
Created PR to 202505 branch
labels
rolin-arista
pushed a commit
to rolin-arista/sonic-mgmt-202505-snappi
that referenced
this pull request
Jul 24, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
nissampa
pushed a commit
to nissampa/sonic-mgmt_dpu_test
that referenced
this pull request
Aug 7, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
ashutosh-agrawal
pushed a commit
to ashutosh-agrawal/sonic-mgmt
that referenced
this pull request
Aug 14, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
vidyac86
pushed a commit
to vidyac86/sonic-mgmt
that referenced
this pull request
Oct 23, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
opcoder0
pushed a commit
to opcoder0/sonic-mgmt
that referenced
this pull request
Dec 8, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
Signed-off-by: opcoder0 <110003254+opcoder0@users.noreply.github.com>
gshemesh2
pushed a commit
to gshemesh2/sonic-mgmt
that referenced
this pull request
Dec 16, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>
AharonMalkin
pushed a commit
to AharonMalkin/sonic-mgmt
that referenced
this pull request
Dec 16, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
Signed-off-by: Aharon Malkin <amalkin@nvidia.com>
gshemesh2
pushed a commit
to gshemesh2/sonic-mgmt
that referenced
this pull request
Dec 21, 2025
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>
venu-nexthop
pushed a commit
to venu-nexthop/sonic-mgmt
that referenced
this pull request
Jan 13, 2026
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
gshemesh2
pushed a commit
to gshemesh2/sonic-mgmt
that referenced
this pull request
Jan 26, 2026
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
Signed-off-by: Guy Shemesh <gshemesh@nvidia.com>
ytzur1
pushed a commit
to ytzur1/sonic-mgmt
that referenced
this pull request
Feb 2, 2026
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
Signed-off-by: Yael Tzur <ytzur@nvidia.com>
kazinator-arista
pushed a commit
to kazinator-arista/sonic-mgmt
that referenced
this pull request
Mar 4, 2026
Why I did it DEFAULT_CONTAINER_REGISTRY didn't work as expected in some scenario. How I did it When check for docker arch, use DEFAULT_CONTAINER_REGISTRY if it is not null.
venu-nexthop
pushed a commit
to venu-nexthop/sonic-mgmt
that referenced
this pull request
Mar 27, 2026
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
> pytest_assert(loss_rate <= max_loss_rate and loss_rate >= min_loss_rate,
'Loss rate of {} ({}) should be in [{}, {}]'.format(
data_flow_name_list[i], loss_rate, min_loss_rate, max_loss_rate))
E Failed: Loss rate of Data Flow 1 (0.5811225531621866) should be in [0.7, 1]
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Bug fix
Testbed and Framework(new/improvement)
New Test case
Skipped for non-supported platforms
Test case improvement
Back port request
202205
202305
202311
202405
202411
202505
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.
signed-off-by: jianquanye@microsoft.com
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
pfcwd_basic tests have to be tuned on platform basis. Since they are timer-sensitive, and can fail in corner cases with the symptom as below:
In this PR, we are increasing the deviation for cisco-8000 to 0.5 instead of the original 0.3.
Type of change
Back port request
Approach
What is the motivation for this PR?
Cisco-8000 runs keep hitting flaky test fails due to above signature.
How did you do it?
Updated the deviation for cisco-8000 to 0.5.
How did you verify/test it?
Run in progress for entire pfcwd_basic. Will update the PR once it is done.
Any platform specific information?
Specific to cisco-8000.