[ACL] Add ACL stress test#15225
Conversation
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
opcoder0
requested review from
kevinskwang,
wangxin,
xwjiang-ms and
yutongzhang-microsoft
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
LGTM |
|
@bingwang-ms pls take a look as well |
tests/run_tests.sh
Outdated
| "test_vlan_ping.py" \ | ||
| "test_auto_techsupport.py" \ | ||
| "test_platform_info.py") | ||
|
|
There was a problem hiding this comment.
Is it a good idea to hard code list of flaky tests here?
There was a problem hiding this comment.
Its been a while this is due to merge issue doesn't belong to this PR. I will have re-merge.
tests/pytest.ini
Outdated
| static_config: static_config marker | ||
| dependency: dependency marker | ||
| skip_traffic_test: skip_traffic_test marker | ||
| stress: stress test marker |
There was a problem hiding this comment.
Is the behavior of pytest stress plugin to run a test many times until the time upper limit is reached?
This way of doing stress test is not efficient enough for our test design. Each of the test script has a large overhead of setting up the global fixtures, do sanity check, etc.
Maybe use the confidence level is more appropriate? You can search the code base for examples of confidence level.
Basically the idea to use in-test-script loop to run something really matters many times. This way can avoid the overhead.
There was a problem hiding this comment.
I will look into it. Unfortunately, there is no way to skip setup and teardown between stress runs. I would think that was the whole idea of the stress test is to do setup, run test N times and then tear down. But pytest-stress doesn't do that. And it appeared to me at that time that I had to manually run it in a loop say 1000 times. I can add an option to pass in the number of loops to run the test; in this case it would translate to number of times the packet is sent/received after the ACL has been setup.
There was a problem hiding this comment.
Thank you @wangxin. I have reverted the reliance on pytest-stress and have relied on using completeness level that other stress tests use.
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
@bingwang-ms can you please review as well. Thanks! |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
The PR adds stress test to ACL. The test generates number rules depending on the hardware SKU and its ACL entry capacity. What is the motivation for this PR? Fixes: sonic-net#3595 How did you do it? Add a new test to perform stress tests. How did you verify/test it? Manually on 7050cx3 and 4600. Any platform specific information? None
The PR adds stress test to ACL. The test generates number rules depending on the hardware SKU and its ACL entry capacity. What is the motivation for this PR? Fixes: sonic-net#3595 How did you do it? Add a new test to perform stress tests. How did you verify/test it? Manually on 7050cx3 and 4600. Any platform specific information? None
r12f
added
Included in msft-202412 Branch
and removed
Created PR to msft-202412 Branch
Cherry Pick Conflict_msft-202412
labels
7 participants
Description of PR
The PR adds stress test to ACL. The test generates number rules depending on the hardware SKU and its ACL entry capacity.
Summary:
Fixes #3595
Type of change
Back port request
Approach
What is the motivation for this PR?
Fixes: #3595
How did you do it?
Add a new test to perform stress tests.
How did you verify/test it?
Manually on 7050cx3 and 4600.
Any platform specific information?
None
Supported testbed topology if it's a new test case?
Documentation
Not applicable