Skip to content

[Test Gap] Cover all cacl protocols in GCU test_cacl#13543

Merged
wangxin merged 2 commits intosonic-net:masterfrom
ZhaohuiS:fix/test_gap_test_cacl_gcu
Jul 11, 2024
Merged

[Test Gap] Cover all cacl protocols in GCU test_cacl#13543
wangxin merged 2 commits intosonic-net:masterfrom
ZhaohuiS:fix/test_gap_test_cacl_gcu

Conversation

@ZhaohuiS
Copy link
Contributor

@ZhaohuiS ZhaohuiS commented Jul 3, 2024

Description of PR

Summary:
Fixes # (issue)
Fix the test gap #12816

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 202012
  • 202205
  • 202305
  • 202311
  • 202405

Approach

What is the motivation for this PR?

Original test_cacl only covers parts of SNMP or SSH protocol test, sonic-net/sonic-host-services#9 introduce EXTERNAL_CLIENT, we have to cover it in GCU test_cacl

How did you do it?

Enhance many functions to support different protocols and add a fixture cacl_protocol to test SSH, SNMP,NTP, EXTERNAL_CLIENT one by one.
Also add T1 topology in mark topology list.

Test steps:
TC1 cacl table:

  1. Test to add a new cacl table, which should expect success
  2. test add duplicated cacl table, which should expect success
  3. Test replace some variable in existing cacl table, which should expect success
  4. Test add invalid cacl table, which should expect failure
  5. Test remove non-existed cacl table, which should expect failure
  6. Test remove cacl table, , which should expect success,
  7. Test previous steps for SSH, SNMP,NTP, EXTERNAL_CLIENT one by one

TC2 cacl rule:

  1. Test to add a new cacl rule, which should expect success and iptables rules are expected
  2. test add duplicated cacl rule, which should expect success and iptables rules are expected
  3. Test replace some variable in existing cacl rule, which should expect success and iptables rules are expected
  4. Test add cacl rule into non-existed cacl table, which should expect failure
  5. Test remove cacl table which has cacl rules, which should expect failure
  6. Test remove non-existed cacl rule, which should expect success,
  7. Test remove ACL_RULE path, which should expect success and none of unexpected iptables rules exists
  8. Test previous steps for SSH, SNMP,NTP, EXTERNAL_CLIENT one by one
  • workitem:

27868222

How did you verify/test it?

collected 8 items                                                                                                                                                                                                                        

generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[SSH] PASSED                                                                                                                                                     [ 12%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[SSH] PASSED                                                                                                                                                       [ 25%]
generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[NTP] PASSED                                                                                                                                                     [ 37%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[NTP] PASSED                                                                                                                                                       [ 50%]
generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[SNMP] PASSED                                                                                                                                                    [ 62%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[SNMP] PASSED                                                                                                                                                      [ 75%]
generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[EXTERNAL_CLIENT] PASSED                                                                                                                                         [ 87%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[EXTERNAL_CLIENT] PASSED                                                                                                                                           [100%]

============================================================================================================ warnings summary ============================================================================================================

Any platform specific information?

Run tests/generic_config_updater/test_cacl.py

Supported testbed topology if it's a new test case?

Documentation

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
Copy link
Contributor

@wen587 wen587 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l'g'tm

@yutongzhang-microsoft
Copy link
Contributor

Hi, @ZhaohuiS, did you test these new added scripts on kvm testbed? Can they be added in PR checker?

@ZhaohuiS
Copy link
Contributor Author

ZhaohuiS commented Jul 9, 2024

Hi, @ZhaohuiS, did you test these new added scripts on kvm testbed? Can they be added in PR checker?

No yet, I don't have kvm test env. But yes, we can add it into PR test, which is only kernel and sw related, not require specific hw.

@yutongzhang-microsoft
Copy link
Contributor

Hi, @ZhaohuiS, did you test these new added scripts on kvm testbed? Can they be added in PR checker?

No yet, I don't have kvm test env. But yes, we can add it into PR test, which is only kernel and sw related, not require specific hw.

Can you try to add in onborading_ set in pr_test_scripts.yaml to test them on kvm?

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
@ZhaohuiS
Copy link
Contributor Author

ZhaohuiS commented Jul 9, 2024

Hi, @ZhaohuiS, did you test these new added scripts on kvm testbed? Can they be added in PR checker?

No yet, I don't have kvm test env. But yes, we can add it into PR test, which is only kernel and sw related, not require specific hw.

Can you try to add in onborading_ set in pr_test_scripts.yaml to test them on kvm?

Done, all of this script cases were passed on PR test on t0 topo.

@mssonicbld
Copy link
Collaborator

@ZhaohuiS PR conflicts with 202405 branch

@mssonicbld
Copy link
Collaborator

@ZhaohuiS PR conflicts with 202311 branch

wangxin pushed a commit that referenced this pull request Jul 18, 2024
Cherry pick #13543 into 202311

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
wangxin pushed a commit that referenced this pull request Jul 18, 2024
Cherry pick #13543 into 202405

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
arista-hpandya pushed a commit to arista-hpandya/sonic-mgmt that referenced this pull request Oct 2, 2024
What is the motivation for this PR?
Original test_cacl only covers parts of SNMP or SSH protocol test, sonic-net/sonic-host-services#9 introduce EXTERNAL_CLIENT, we have to cover it in GCU test_cacl

How did you do it?
Enhance many functions to support different protocols and add a fixture cacl_protocol to test SSH, SNMP,NTP, EXTERNAL_CLIENT one by one.
Also add T1 topology in mark topology list.

Test steps:
TC1 cacl table:
1. Test to add a new cacl table, which should expect success
2. test add duplicated cacl table, which should expect success
3. Test replace some variable in existing cacl table, which should expect success
4. Test add invalid cacl table, which should expect failure
5. Test remove non-existed cacl table, which should expect failure
6. Test remove cacl table, , which should expect success,
7. Test previous steps for SSH, SNMP,NTP, EXTERNAL_CLIENT one by one

TC2 cacl rule:
1. Test to add a new cacl rule, which should expect success and iptables rules are expected
2. test add duplicated cacl rule, which should expect success and iptables rules are expected
3. Test replace some variable in existing cacl rule, which should expect success and iptables rules are expected
4. Test add cacl rule into non-existed cacl table, which should expect failure
5. Test remove cacl table which has cacl rules, which should expect failure
6. Test remove non-existed cacl rule, which should expect success,
7. Test remove ACL_RULE path, which should expect success and none of unexpected iptables rules exists
8. Test previous steps for SSH, SNMP,NTP, EXTERNAL_CLIENT one by one

How did you verify/test it?
collected 8 items                                                                                                                                                                                                                        

generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[SSH] PASSED                                                                                                                                                     [ 12%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[SSH] PASSED                                                                                                                                                       [ 25%]
generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[NTP] PASSED                                                                                                                                                     [ 37%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[NTP] PASSED                                                                                                                                                       [ 50%]
generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[SNMP] PASSED                                                                                                                                                    [ 62%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[SNMP] PASSED                                                                                                                                                      [ 75%]
generic_config_updater/test_cacl.py::test_cacl_tc1_acl_table_suite[EXTERNAL_CLIENT] PASSED                                                                                                                                         [ 87%]
generic_config_updater/test_cacl.py::test_cacl_tc2_acl_rule_test[EXTERNAL_CLIENT] PASSED                                                                                                                                           [100%]

============================================================================================================ warnings summary ============================================================================================================
Any platform specific information?
Run tests/generic_config_updater/test_cacl.py

Signed-off-by: Zhaohui Sun <zhaohuisun@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants