SONiC switch High-scale IPv6 BGP test plan by sm-xu · Pull Request #16759 · sonic-net/sonic-mgmt

sm-xu · 2025-02-03T00:53:27Z

Description of PR

Summary:
Fixes # (issue)

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

mssonicbld · 2025-02-03T00:53:30Z

/azp run

azure-pipelines · 2025-02-03T00:53:41Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-02-03T01:23:18Z

/azp run

azure-pipelines · 2025-02-03T01:23:26Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

r12f · 2025-02-08T02:18:33Z

docs/testplan/BGP_IPv6_test_2TierNetwork.md

@@ -0,0 +1,32 @@
+# Test Objective
+This test aims to verify the scalability and stability of 256 BGP sessions and 10K IPv6 routes in a 2-tier network. It evaluates the DUT’s ability to establish and maintain BGP sessions, ensures proper route learning, and measures BGP update convergence time under various conditions.


setup is 1 BGP session per port.

so we are not limited to 256 BGP sessions

r12f · 2025-02-08T02:20:03Z

docs/testplan/BGP_IPv6_test_2TierNetwork.md

@@ -0,0 +1,32 @@
+# Test Objective
+This test aims to verify the scalability and stability of 256 BGP sessions and 10K IPv6 routes in a 2-tier network. It evaluates the DUT’s ability to establish and maintain BGP sessions, ensures proper route learning, and measures BGP update convergence time under various conditions.


mutli-tier network should still work.

r12f · 2025-02-08T02:20:29Z

docs/testplan/BGP_IPv6_test_2TierNetwork.md

+This test aims to verify the scalability and stability of 256 BGP sessions and 10K IPv6 routes in a 2-tier network. It evaluates the DUT’s ability to establish and maintain BGP sessions, ensures proper route learning, and measures BGP update convergence time under various conditions.
+
+# Test Setup
+![Test Setup](./2TierNetwork.png)


nit: lint the markdown.

r12f · 2025-02-08T02:20:58Z

docs/testplan/BGP_IPv6_test_2TierNetwork.md

@@ -0,0 +1,32 @@
+# Test Objective


BGP + route scale

actually, better split the route scale into another test. for bgp session, we can use 4 x Number of ports as route scale.

for route scale test, we can do 40 x number of ports. but in separate test. in the route scale test, we need to check the latency very carefully.

r12f · 2025-02-08T02:57:05Z

docs/testplan/BGP_IPv6_test_2TierNetwork.md

+# Test Setup
+![Test Setup](./2TierNetwork.png)
+
+1.	The testbed consists of four IXIA traffic generators (synchronized using a time-sync metronome) and five SONiC switches, where the BT1 switch is the Device Under Test (DUT).


the test should work with any number of ixias.

r12f · 2025-02-08T02:58:26Z

docs/testplan/BGP_IPv6_test_2TierNetwork.md

+5.	The routing configuration of the BT0 switches should ensure that all data traffic go through the DUT.
+
+# Test Steps
+1. Assign a unique AS number to each of the five switches.


we should consider automate this using add-topo or deploy-mg.

all switches can share the same setup, and ask the IXIA to advertise the routes.

mssonicbld · 2025-02-11T22:50:00Z

/azp run

azure-pipelines · 2025-02-11T22:50:07Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

r12f · 2025-02-15T00:09:00Z

docs/testplan/BGP_IPv6_test.md

+
+2. Between each of the four neighboring switches and the DUT: Configure X/Y BGP sessions. Each BGP session should have a dedicated pair of Ethernet ports (one on the DUT and the other on the neighboring device) whose IPv6 addresses are on the same subnet. Set up the BGP neighbors, device neighbors, and port IPv6 addresses for each BGP session.
+
+3. Monitor the BGP session establishment on the DUT using command “show ipv6 bgp summary”. Ensure all X BGP sessions are established without errors.


command should be quoted by `

r12f · 2025-02-15T00:15:07Z

docs/testplan/BGP_IPv6_test.md

+
+3. Monitor the BGP session establishment on the DUT using command “show ipv6 bgp summary”. Ensure all X BGP sessions are established without errors.
+
+4. In each neighboring switch: Configure a vlan,  assign 2500 IPv6 addresses with the specified prefix length and add all the Ethernet ports connected to IXIA to the vlan.


the 2500 should be total number of port x10 (considering 1 port, 1 VLAN route)

r12f · 2025-02-15T00:16:31Z

docs/testplan/BGP_IPv6_test.md

+
+In the above example, the DUT has 256 logical Ethernet ports and is connected to 4 neighboring switches, we will establish 64 BGP sessions between each neighbor and the DUT.
+
+## Test Steps


we should have 3 test cases:

All session shutdown/enable time

1 session shutdown/enable time

Fragmented failure links / Nexthop Group Member Scale Test

see this: https://github.com/sonic-net/sonic-mgmt/pull/15702/files

The 1st and 2nd cases are done. Working on the 3rd case...

Re-wrote the 3rd case.

mssonicbld · 2025-02-18T19:46:41Z

/azp run

azure-pipelines · 2025-02-18T19:46:49Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

mssonicbld · 2025-04-02T19:50:40Z

/azp run

azure-pipelines · 2025-04-02T19:50:48Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

r12f · 2025-04-03T05:30:36Z

docs/testplan/BGP_IPv6_test.md

+1. In one of the T0 switches, run `show ipv6 bgp network <ipv6>/<prefix>` and find the number of nexthops that can be used to reach `<ipv6>/<prefix>`.
+2. Randomly pick half of the next hops and remove them. Run the show command again and record the convergence time.
+3. Restore the removed nexthops and record the convergence time again.
+4. Repeat this process and calculate the average convergence time of this scenario.


missing the metrics definition here.

Added. Thank you!

mssonicbld · 2025-04-14T03:45:26Z

/azp run

azure-pipelines · 2025-04-14T03:45:33Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

r12f · 2025-04-16T03:35:09Z

docs/testplan/BGP_IPv6_test.md

+
+## Test Steps
+
+1. Assign a unique AS number to each of the five switches.


The ASN should be coming from topology. we should mention this in the test setup section as topology introduction.

Moved to test setup section.

r12f · 2025-04-16T03:37:51Z

docs/testplan/BGP_IPv6_test.md

+
+1. Assign a unique AS number to each of the five switches.
+
+2. Between each of the neighboring switches and the DUT: Configure X/Y BGP sessions. Each BGP session should have a dedicated pair of Ethernet ports (one on the DUT and the other on the neighboring device) whose IPv6 addresses are on the same subnet. Set up the BGP neighbors, device neighbors, and port IPv6 addresses for each BGP session.


with our current setup of multi-tier topology, we should not have any special setup for this one.

however, in the test setup, we can mention the approach we use to stress the BGP session on a device.

e.g. with multiple T0 and a single T1 being used.

I revised both test plans. Please review.

r12f · 2025-04-16T03:39:11Z

docs/testplan/BGP_IPv6_test.md

+
+2. Between each of the neighboring switches and the DUT: Configure X/Y BGP sessions. Each BGP session should have a dedicated pair of Ethernet ports (one on the DUT and the other on the neighboring device) whose IPv6 addresses are on the same subnet. Set up the BGP neighbors, device neighbors, and port IPv6 addresses for each BGP session.
+
+3. Monitor the BGP session establishment on the DUT using command `show ipv6 bgp summary`. Ensure all X BGP sessions are established without errors.


this step should be the first pretest step.

r12f · 2025-04-16T03:40:07Z

docs/testplan/BGP_IPv6_test.md

+
+3. Monitor the BGP session establishment on the DUT using command `show ipv6 bgp summary`. Ensure all X BGP sessions are established without errors.
+
+4. In each neighboring switch: Configure a vlan, assign `10*X` IPv6 addresses with the specified prefix length and add all the Ethernet ports connected to IXIA to the vlan.


might need to update this step with ixia advertising the routes?

r12f · 2025-04-16T03:41:55Z

docs/testplan/BGP_IPv6_test.md

+
+## Key Test Cases
+
+### One BGP Session Flap


### Case 1: ...

r12f · 2025-04-16T03:43:41Z

docs/testplan/BGP_IPv6_test.md

+
+### One BGP Session Flap
+
+1. One session down and up: Shut down one interface on the DUT. Wait till all routes advertised by the impacted BGP session are removed.


Port shutdown
Step 1: Start traffic
Step 2: Shutdown port
Step 3: Evaluate data path reaction time
Step 4: Evaluate route update time

repeat reverse for port startup

we need to be precise on how to evaluate, which command to run and etc.

Please check my revised version.

r12f · 2025-04-16T03:45:39Z

docs/testplan/BGP_IPv6_test.md

+
+### All BGP Sessions Down and Up Test
+
+1. Stop the BGP container on the DUT. Wait till all BGP routes are removed.


same as above. need to make it better and also we need to focus more on the data plane reaction time, since we have ixia in our testbed.

Please check out my Teams message.

r12f · 2025-04-16T03:48:04Z

docs/testplan/BGP_IPv6_test.md

+
+| Metric Name                                     | Example Value       |
+| ----------------------------------------------- | ------------------- |
+| `METRIC_NAME_BGP_CONVERGENCE_PORT_RESTART`      | 15                  |


test.bgp_scale.one_port_down.route_convergence_time_ms
test.bgp_scale.one_port_down.dp_response_time_ms
test.bgp_scale.all_port_down.route_convergence_time_ms
....

Convergency time is measured in seconds, not milliseconds. Using a suffix like *_s, might cause confusion about what that is.

mssonicbld · 2025-04-22T04:11:39Z

/azp run

azure-pipelines · 2025-04-22T04:11:47Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

docs/testplan/BGP_IPv6_test.md

docs/testplan/example_layout.png

mssonicbld · 2025-05-13T04:14:07Z

/azp run

azure-pipelines · 2025-05-13T04:14:18Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-05-13T04:18:17Z

/azp run

azure-pipelines · 2025-05-13T04:18:25Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

mssonicbld · 2025-05-13T05:45:45Z

/azp run

azure-pipelines · 2025-05-13T05:45:53Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

mssonicbld · 2025-05-13T05:54:38Z

/azp run

azure-pipelines · 2025-05-13T05:54:45Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

r12f · 2025-07-13T07:03:25Z

Close in favor of #19564

sm-xu added 4 commits January 16, 2025 15:58

1st version of interface utils

876ce09

Merge remote-tracking branch 'upstream/master'

75a90f3

Merge remote-tracking branch 'upstream/master'

1c5a0da

BGP test doc 02/02

25e0f53

sm-xu requested review from wangxin and yxieca as code owners February 3, 2025 00:53

remove a deprecated file

dd47aa1

Merge remote-tracking branch 'upstream/master'

6f5a59a

sm-xu changed the title ~~2-tier network IPv6 BGP test plan~~ Multi-tier network IPv6 BGP test plan Feb 6, 2025

r12f reviewed Feb 8, 2025

View reviewed changes

sm-xu added 2 commits February 11, 2025 20:13

Merge remote-tracking branch 'upstream/master'

87f90ed

update 02/11

7318ca6

sm-xu changed the title ~~Multi-tier network IPv6 BGP test plan~~ SONiC switch High-scale IPv6 BGP test plan Feb 11, 2025

r12f reviewed Feb 15, 2025

View reviewed changes

update 02/18

445c786

sm-xu added 3 commits February 23, 2025 01:37

Merge remote-tracking branch 'upstream/master'

9125b28

Merge remote-tracking branch 'upstream/master'

fbe30bb

update 04/02

6cbdff0

r12f reviewed Apr 3, 2025

View reviewed changes

update 04/13

0d8f40a

r12f reviewed Apr 16, 2025

View reviewed changes

update 04/21

c723d49

r12f reviewed May 1, 2025

View reviewed changes

docs/testplan/BGP_IPv6_test.md Show resolved Hide resolved

r12f reviewed May 11, 2025

View reviewed changes

docs/testplan/BGP_IPv6_test.md Show resolved Hide resolved

r12f reviewed May 11, 2025

View reviewed changes

docs/testplan/example_layout.png Show resolved Hide resolved

sm-xu added 6 commits May 13, 2025 03:51

addressed review comments

7be75c9

Merge branch 'master' of github.com:sm-xu/sonic-mgmt-msr

bc1a42e

Merge branch 'master' into bgp-test-doc

9a2add0

1st version of interface utils

725f248

Merge branch 'master' into bgp-test-doc

c89dc01

move the test plan to snappi-tests folder

9abbb48

remove a deprecated folder

843a9d6

add a doc link

a2fe59c

add an example

15a7a10

r12f closed this Jul 13, 2025

		@@ -0,0 +1,32 @@
		# Test Objective
		This test aims to verify the scalability and stability of 256 BGP sessions and 10K IPv6 routes in a 2-tier network. It evaluates the DUT’s ability to establish and maintain BGP sessions, ensures proper route learning, and measures BGP update convergence time under various conditions.


		2. Between each of the four neighboring switches and the DUT: Configure X/Y BGP sessions. Each BGP session should have a dedicated pair of Ethernet ports (one on the DUT and the other on the neighboring device) whose IPv6 addresses are on the same subnet. Set up the BGP neighbors, device neighbors, and port IPv6 addresses for each BGP session.

		3. Monitor the BGP session establishment on the DUT using command “show ipv6 bgp summary”. Ensure all X BGP sessions are established without errors.


		3. Monitor the BGP session establishment on the DUT using command “show ipv6 bgp summary”. Ensure all X BGP sessions are established without errors.

		4. In each neighboring switch: Configure a vlan, assign 2500 IPv6 addresses with the specified prefix length and add all the Ethernet ports connected to IXIA to the vlan.


		In the above example, the DUT has 256 logical Ethernet ports and is connected to 4 neighboring switches, we will establish 64 BGP sessions between each neighbor and the DUT.

		## Test Steps


		## Test Steps

		1. Assign a unique AS number to each of the five switches.


		1. Assign a unique AS number to each of the five switches.

		2. Between each of the neighboring switches and the DUT: Configure X/Y BGP sessions. Each BGP session should have a dedicated pair of Ethernet ports (one on the DUT and the other on the neighboring device) whose IPv6 addresses are on the same subnet. Set up the BGP neighbors, device neighbors, and port IPv6 addresses for each BGP session.


		### One BGP Session Flap

		1. One session down and up: Shut down one interface on the DUT. Wait till all routes advertised by the impacted BGP session are removed.


		### All BGP Sessions Down and Up Test

		1. Stop the BGP container on the DUT. Wait till all BGP routes are removed.

Conversation

sm-xu commented Feb 3, 2025

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

mssonicbld commented Feb 3, 2025

Uh oh!

azure-pipelines bot commented Feb 3, 2025

Uh oh!

mssonicbld commented Feb 3, 2025

Uh oh!

azure-pipelines bot commented Feb 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Feb 11, 2025

Uh oh!

azure-pipelines bot commented Feb 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Feb 18, 2025

Uh oh!

azure-pipelines bot commented Feb 18, 2025

Uh oh!

mssonicbld commented Apr 2, 2025

Uh oh!

azure-pipelines bot commented Apr 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Apr 14, 2025

Uh oh!

azure-pipelines bot commented Apr 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment