Skip to content

Commit c912484

Browse files
yxiecayxieca
authored
[test plans] Moving test plans from sonic-wiki repo to sonic-mgmt repo (#1631)
Signed-off-by: Ying Xie <[email protected]>
1 parent 8e608c3 commit c912484

8 files changed

+1727
-0
lines changed

‎docs/ACL-test-plan.md‎

Lines changed: 426 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/BGP-GR-helper-mode-test-plan.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
- [Overview](#overview)
2+
- [Scope](#scope)
3+
- [Testbed](#testbed)
4+
- [Setup configuration](#setup-configuration)
5+
- [Arista VM configuration](#arista-vm-configuration)
6+
- [Ansible scripts to setup and run test](#ansible-scripts-to-setup-and-run-test)
7+
- [everflow_testbed.yml](#everflow-testbed-yml)
8+
- [PTF Test](#ptf-test)
9+
- [Input files for PTF test](#input-files-for-ptf-test)
10+
- [Traffic validation in PTF](#traffic-validation-in-ptf)
11+
- [Test cases](#test-cases)
12+
- [TODO](#todo)
13+
- [Open Questions](#open-questions)
14+
15+
## Overview
16+
The purpose is to test a functionality of BGP GR mode on the SONIC switch DUT, closely resembling production environment.
17+
The test assumes all necessary configuration is already pre-configured on the SONIC switch before test runs.
18+
19+
### Scope
20+
The test is targeting a running SONIC system with fully functioning configuration.
21+
The purpose of the test is not to test specific API, but functional testing of BGP GR helper mode on SONIC system, making sure that traffic flows correctly, according to BGP routes advertised by BGP peers of SONIC switch.
22+
23+
### Testbed
24+
The test will run on the following testbeds:
25+
- t1
26+
- t1-lag
27+
28+
## Setup configuration
29+
30+
#### Arista VM configuration
31+
32+
Test assumes that BGP GR is enabled and preconfigured on Arista VMs. BGP GR timer value should be more than time required for VM reboot.
33+
34+
#### Ansible scripts to setup and run test
35+
36+
##### bgp_gr_helper.yml
37+
38+
bgp_gr_helper.yml when run with tag "bgp_gr_helper" will do the following:
39+
40+
1. Randomly choose VM.
41+
2. Run test.
42+
43+
BGP GR helper test consists of a number of subtests, and each of them will include the following steps:
44+
45+
1. Run lognanalyzer 'init' phase
46+
2. Run BGP GR helper Sub Test
47+
3. Run loganalyzer 'analyze' phase
48+
49+
## PTF Test
50+
51+
To run traffic FIB PTF test will be reused.
52+
53+
## Test cases
54+
55+
Each test case will be additionally validated by the loganalizer utility.
56+
57+
### Test case \#1 - BGP GR helper mode.
58+
59+
#### Test objective
60+
61+
Verify that routes are preserved during neighbor graceful restart.
62+
63+
#### Test steps
64+
65+
- Randomly choose VM for the test.
66+
- Reboot VM.
67+
- Verify BGP timeout (at least 115 seconds routes should stay in fib).
68+
- Verify all routes are preserved (no reinstallation after BGP open message from the neighbor).
69+
- Verify that BGP session with the VM established.
70+
71+
### Test case \#2 - BGP GR helper mode routes change.
72+
73+
#### Test objective
74+
75+
Verify that traffic run without changes during neighbor graceful restart.
76+
77+
#### Test steps
78+
79+
- Randomly choose VM for the test.
80+
- Change VM startup config (advertised routes should be different).
81+
- Reboot VM.
82+
- Verify that preserved routes are removed when VM back.
83+
- Verify that new routes are installed when VM back.
84+
- Restore VM startup config.
85+
86+
## TODO
87+
88+
## Open Questions
89+
- Should tests run for neighbors behind physical interfaces only or behind LAGs as well?
90+
- On which topologies test should run?

‎docs/BGP-MP-test-plan.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# BGP-MP test plan
2+
3+
* [Overview](#Overview)
4+
* [Scope](#Scope)
5+
* [Testbed](#Testbed)
6+
* [Setup configuration](#Setup%20configuration)
7+
* [Ansible scripts to setup and run test](#Ansible%20scripts%20to%20setup%20and%20run%20test)
8+
* [bgp_mp.yml](#bgp_mp.yml)
9+
* [Setup of DUT switch](#Setup%20of%20DUT%20switch)
10+
* [Test](#Test)
11+
* [Test cases](#Test%20cases)
12+
* [TODO](#TODO)
13+
* [Open questions](#Open%20questions)
14+
15+
## Overview
16+
The purpose is to test functionality of BGP-MP on the SONIC switch DUT, closely resembling production environment. The test assumes all necessary configurations are already pre-configured on the SONIC switch before test runs.
17+
18+
### Scope
19+
The test is targeting a running SONIC system with fully functioning configuration. The purpose of the test is not to test specific API, but functional testing of BGP-MP on SONIC system.
20+
21+
### Testbed
22+
The test will run on the following testbeds:
23+
* t0
24+
25+
## Setup configuration
26+
IPv4 BGP neighborship will be configured between DUT and exabgp and each neighbor will redistribute IPv6 routes to each other.
27+
### Ansible scripts to setup and run test
28+
#### bgp_mp.yml
29+
bgp_mp.yml when run with tag “bgp_mp” will do the following:
30+
1. Generate and apply exabgp configuration.
31+
2. Run test.
32+
3. Clean up dynamic and temporary exabgp configuration.
33+
34+
## Test
35+
On PTF host, exabgp tool will be used to configure bgp peer and redistribute IPv6 routes via IPv4 BGP session.
36+
37+
## Test cases
38+
### Test case # 1 – BGP-MP IPv6 routes over IPv4 session
39+
#### Test objective
40+
Verify that IPv6 routes are correctly redistributed over IPv4 BGP session.
41+
#### Test steps
42+
* Generate IPv4 BGP peer configuration for exabgp instance.
43+
* Generate IPv6 routes, to be announced via IPv4 session, for exabgp instance.
44+
* Run exabgp instance.
45+
* Verify that IPv4 BGP neighborship is established.
46+
* Redistribute IPv6 routes using exabgp.
47+
* Verify that IPv6 routes are correctly redistributed to the DUT.
48+
* Redistribute IPv6 routes from the DUT to exabgp.
49+
* Verify that IPv6 routes are correctly redistributed to the exabgp.
50+
* Set default configuration.
51+
52+
## TODO
53+
54+
## Open questions
55+
* Should be some traffic test cases performed as part of this test?

‎docs/CRM-test-plan.md‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
# CRM test plan
2+
3+
* [Overview](#Overview)
4+
* [Scope](#Scope)
5+
* [Testbed](#Testbed)
6+
* [Setup configuration](#Setup%20configuration)
7+
* [Ansible scripts to setup and run test](#Ansible%20scripts%20to%20setup%20and%20run%20test)
8+
* [crm.yml](#crm.yml)
9+
* [Test](#Test)
10+
* [Test cases](#Test%20cases)
11+
* [TODO](#TODO)
12+
* [Open questions](#Open%20questions)
13+
14+
## Overview
15+
The purpose is to test functionality of CRM on the SONIC switch DUT, closely resembling production environment.
16+
17+
### Scope
18+
The test is targeting a running SONIC system with fully functioning configuration. The purpose of the test is not to test specific API, but functional testing of CRM on SONIC system.
19+
20+
### Testbed
21+
The test will run on the all testbeds.
22+
23+
## Setup configuration
24+
No setup pre-configuration is required, test will configure and clean-up all the configuration.
25+
### Ansible scripts to setup and run test
26+
#### crm.yml
27+
crm.yml when run with tag “crm” will do the following for each CRM resource:
28+
1. Apply required configuration.
29+
2. Verify "used" and "free" counters.
30+
3. Verify "EXCEEDED" and "CLEAR" messages using all types of thresholds.
31+
4. Restore configuration.
32+
33+
## Test
34+
35+
## Test cases
36+
37+
### Test case # 1 – IPv4 route
38+
#### Test objective
39+
Verify "IPv4 route" CRM resource.
40+
#### Test steps
41+
* Set polling interval to 1 minute.
42+
* Configure 1 route and observe that counters were updated as expected.
43+
* Remove 1 route and observe that counters were updated as expected.
44+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
45+
* Set low and high thresholds according to current usage and type.
46+
* Verify that "EXCEEDED" message is logged (using log analyzer).
47+
* Set low and high thresholds to default values.
48+
* Verify that "CLEAR" message is logged (using log analyzer).
49+
* Restore default configuration.
50+
51+
### Test case # 2 – IPv6 route
52+
#### Test objective
53+
Verify "IPv6 route" CRM resource.
54+
#### Test steps
55+
* Set polling interval to 1 minute.
56+
* Configure 1 route and observe that counters were updated as expected.
57+
* Remove 1 route and observe that counters were updated as expected.
58+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
59+
* Set low and high thresholds according to current usage and type.
60+
* Verify that "EXCEEDED" message is logged (using log analyzer).
61+
* Set low and high thresholds to default values.
62+
* Verify that "CLEAR" message is logged (using log analyzer).
63+
* Restore default configuration.
64+
65+
### Test case # 3 – IPv4 nexthop
66+
#### Test objective
67+
Verify "IPv4 nexthop" CRM resource.
68+
#### Test steps
69+
* Set polling interval to 1 minute.
70+
* Add 1 nexthop and observe that counters were updated as expected.
71+
* Remove 1 nexthop and observe that counters were updated as expected.
72+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
73+
* Set low and high thresholds according to current usage and type.
74+
* Verify that "EXCEEDED" message is logged (using log analyzer).
75+
* Set low and high thresholds to default values.
76+
* Verify that "CLEAR" message is logged (using log analyzer).
77+
* Restore default configuration.
78+
79+
### Test case # 4 – IPv6 nexthop
80+
#### Test objective
81+
Verify "IPv6 nexthop" CRM resource.
82+
#### Test steps
83+
* Set polling interval to 1 minute.
84+
* Add 1 nexthop and observe that counters were updated as expected.
85+
* Remove 1 nexthop and observe that counters were updated as expected.
86+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
87+
* Set low and high thresholds according to current usage and type.
88+
* Verify that "EXCEEDED" message is logged (using log analyzer).
89+
* Set low and high thresholds to default values.
90+
* Verify that "CLEAR" message is logged (using log analyzer).
91+
* Restore default configuration.
92+
93+
### Test case # 5 – IPv4 neighbor
94+
#### Test objective
95+
Verify "IPv4 neighbor" CRM resource.
96+
#### Test steps
97+
* Set polling interval to 1 minute.
98+
* Configure 1 neighbor and observe that counters were updated as expected.
99+
* Remove 1 neighbor and observe that counters were updated as expected.
100+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
101+
* Set low and high thresholds according to current usage and type.
102+
* Verify that "EXCEEDED" message is logged (using log analyzer).
103+
* Set low and high thresholds to default values.
104+
* Verify that "CLEAR" message is logged (using log analyzer).
105+
* Restore default configuration.
106+
107+
### Test case # 6 – IPv6 neighbor
108+
#### Test objective
109+
Verify "IPv6 neighbor" CRM resource.
110+
#### Test steps
111+
* Set polling interval to 1 minute.
112+
* Configure 1 neighbor and observe that counters were updated as expected.
113+
* Remove 1 neighbor and observe that counters were updated as expected.
114+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
115+
* Set low and high thresholds according to current usage and type.
116+
* Verify that "EXCEEDED" message is logged (using log analyzer).
117+
* Set low and high thresholds to default values.
118+
* Verify that "CLEAR" message is logged (using log analyzer).
119+
* Restore default configuration.
120+
121+
### Test case # 7 – Nexthop group object
122+
#### Test objective
123+
Verify "nexthop group object" CRM resource.
124+
#### Test steps
125+
* Set polling interval to 1 minute.
126+
* Configure 1 ECMP route and observe that counters were updated as expected.
127+
* Remove 1 ECMP route and observe that counters were updated as expected.
128+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
129+
* Set low and high thresholds according to current usage and type.
130+
* Verify that "EXCEEDED" message is logged (using log analyzer).
131+
* Set low and high thresholds to default values.
132+
* Verify that "CLEAR" message is logged (using log analyzer).
133+
* Restore default configuration.
134+
135+
### Test case # 8 – Nexthop group member
136+
#### Test objective
137+
Verify "nexthop group member" CRM resource.
138+
#### Test steps
139+
* Set polling interval to 1 minute.
140+
* Configure 1 ECMP route and observe that counters were updated as expected.
141+
* Remove 1 ECMP route and observe that counters were updated as expected.
142+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
143+
* Set low and high thresholds according to current usage and type.
144+
* Verify that "EXCEEDED" message is logged (using log analyzer).
145+
* Set low and high thresholds to default values.
146+
* Verify that "CLEAR" message is logged (using log analyzer).
147+
* Restore default configuration.
148+
149+
### Test case # 9 – FDB entry
150+
#### Test objective
151+
Verify "FDB entry" CRM resource.
152+
#### Test steps
153+
* Set polling interval to 1 minute.
154+
* Configure 1 FDB entry and observe that counters were updated as expected.
155+
* Remove 1 FDB entry and observe that counters were updated as expected.
156+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
157+
* Set low and high thresholds according to current usage and type.
158+
* Verify that "EXCEEDED" message is logged (using log analyzer).
159+
* Set low and high thresholds to default values.
160+
* Verify that "CLEAR" message is logged (using log analyzer).
161+
* Restore default configuration.
162+
163+
### Test case # 10 – ACL group
164+
#### Test objective
165+
Verify "ACL group" CRM resource.
166+
#### Test steps
167+
* Set polling interval to 1 minute.
168+
* Configure 1 ACL and observe that counters were updated as expected.
169+
* Remove 1 ACL and observe that counters were updated as expected.
170+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
171+
* Set low and high thresholds according to current usage and type.
172+
* Verify that "EXCEEDED" message is logged (using log analyzer).
173+
* Set low and high thresholds to default values.
174+
* Verify that "CLEAR" message is logged (using log analyzer).
175+
* Restore default configuration.
176+
177+
### Test case # 11 – ACL table
178+
#### Test objective
179+
Verify "ACL table" CRM resource.
180+
#### Test steps
181+
* Set polling interval to 1 minute.
182+
* Configure 1 ACL and observe that counters were updated as expected.
183+
* Remove 1 ACL and observe that counters were updated as expected.
184+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
185+
* Set low and high thresholds according to current usage and type.
186+
* Verify that "EXCEEDED" message is logged (using log analyzer).
187+
* Set low and high thresholds to default values.
188+
* Verify that "CLEAR" message is logged (using log analyzer).
189+
* Restore default configuration.
190+
191+
### Test case # 12 – ACL entry
192+
#### Test objective
193+
Verify "ACL entry" CRM resource.
194+
#### Test steps
195+
* Set polling interval to 1 minute.
196+
* Configure 1 ACL rule and observe that counters were updated as expected.
197+
* Remove 1 ACL rule and observe that counters were updated as expected.
198+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
199+
* Set low and high thresholds according to current usage and type.
200+
* Verify that "EXCEEDED" message is logged (using log analyzer).
201+
* Set low and high thresholds to default values.
202+
* Verify that "CLEAR" message is logged (using log analyzer).
203+
* Restore default configuration.
204+
205+
### Test case # 13 – ACL counter
206+
#### Test objective
207+
Verify "ACL entry" CRM resource.
208+
#### Test steps
209+
* Set polling interval to 1 minute.
210+
* Configure 1 ACL rule and observe that counters were updated as expected.
211+
* Remove 1 ACL rule and observe that counters were updated as expected.
212+
* Perform the following steps for all threshold types ("percentage", "used", "free"):
213+
* Set low and high thresholds according to current usage and type.
214+
* Verify that "EXCEEDED" message is logged (using log analyzer).
215+
* Set low and high thresholds to default values.
216+
* Verify that "CLEAR" message is logged (using log analyzer).
217+
* Restore default configuration.
218+
219+
## TODO
220+
221+
## Open questions

0 commit comments

Comments
 (0)