[ssw][ha] introducing `custom_bfd` and BfdMonitorOrch to support primary/secondary NHG switchover by zjswhhh · Pull Request #3922 · sonic-net/sonic-swss

zjswhhh · 2025-10-08T15:19:00Z

What I did
Today vnetorch only supports:

custom monitoring sessions with primary & secondary NHG.
bfd monitoring with primary NHG only.

To make HA work we need bfd monitoring to support primary+secondary. This PR is to introduce a new monitoring type custom_bfd for that purpose. It will:

create bfd sessions for both primary & secondary endpoints
listen on state db for bfd session state update
ignore update call from bfdorch
switch and update nhg based on bfd session state

It's not an elegant design to introduce a new bfd monitoring type. But the intention is to not break current functionality of vnetorch. Refactor of the code will be needed in future.

sign-off: Jing Zhang zhangjing@microsoft.com

Why I did it
To support HA.

How I verified it
Tested on smartswitch.

hamgrd populated APPL_DB table below

monitoring should be custom_bfd.

admin@********:~$ redis-cli -n 0 hgetall "VNET_ROUTE_TUNNEL_TABLE:Vnet_55:3.2.1.0"
 1) "tx_monitor_timer"
 2) "1000"
 3) "check_directly_connected"
 4) "true"
 5) "endpoint_monitor"
 6) "20.0.200.0,20.0.201.0"
 7) "primary"
 8) "20.0.200.0"
 9) "endpoint"
10) "20.0.200.0,10.1.1.32"
11) "rx_monitor_timer"
12) "1000"
13) "monitoring"
14) "custom_bfd"

BFD sessions are created for each endpoint. Both sessions are down, so no nexthop.

admin@********:~$ show bfd sum 
Total number of BFD sessions: 2
Peer Addr    Interface    Vrf      State    Type          Local Addr      TX Interval    RX Interval    Multiplier  Multihop      Local Discriminator
-----------  -----------  -------  -------  ------------  ------------  -------------  -------------  ------------  ----------  ---------------------
20.0.200.0   default      default  Down     async_active  10.1.0.32              1000           1000            10  true                            1
20.0.201.0   default      default  Down     async_active  10.1.0.32              1000           1000            10  true                            2
admin@********:~$ redis-cli -n 6 hgetall "VNET_ROUTE_TUNNEL_TABLE|Vnet_55|3.2.1.0/32"
1) "active_endpoints"
2) ""
3) "state"
4) "inactive"

Set primary as up. Nexthop pointed to primary.

admin@********:~$ show bfd sum 
Total number of BFD sessions: 2
Peer Addr    Interface    Vrf      State    Type          Local Addr      TX Interval    RX Interval    Multiplier  Multihop      Local Discriminator
-----------  -----------  -------  -------  ------------  ------------  -------------  -------------  ------------  ----------  ---------------------
20.0.200.0   default      default  Up       async_active  10.1.0.32              1000           1000            10  true                            1
20.0.201.0   default      default  Down     async_active  10.1.0.32              1000           1000            10  true                            2
admin@********:~$  redis-cli -n 6 hgetall "VNET_ROUTE_TUNNEL_TABLE|Vnet_55|3.2.1.0/32"
1) "active_endpoints"
2) "20.0.200.0"
3) "state"
4) "active"

Set secondary as up. Still pointed to primary.

admin@********:~$ redis-cli -n 6 hset  "BFD_SESSION_TABLE|default|default|20.0.201.0" state Up
(integer) 0
admin@********:~$ show bfd sum 
Total number of BFD sessions: 2
Peer Addr    Interface    Vrf      State    Type          Local Addr      TX Interval    RX Interval    Multiplier  Multihop      Local Discriminator
-----------  -----------  -------  -------  ------------  ------------  -------------  -------------  ------------  ----------  ---------------------
20.0.200.0   default      default  Up       async_active  10.1.0.32              1000           1000            10  true                            1
20.0.201.0   default      default  Up       async_active  10.1.0.32              1000           1000            10  true                            2
admin@********:~$ redis-cli -n 6 hgetall "VNET_ROUTE_TUNNEL_TABLE|Vnet_55|3.2.1.0/32"
1) "active_endpoints"
2) "20.0.200.0"
3) "state"
4) "active"

Set primary down. Nexthop pointed to secondary.

admin@********:~$ redis-cli -n 6 hset  "BFD_SESSION_TABLE|default|default|20.0.200.0" state Down
(integer) 0
admin@********:~$ show bfd sum 
Total number of BFD sessions: 2
Peer Addr    Interface    Vrf      State    Type          Local Addr      TX Interval    RX Interval    Multiplier  Multihop      Local Discriminator
-----------  -----------  -------  -------  ------------  ------------  -------------  -------------  ------------  ----------  ---------------------
20.0.200.0   default      default  Down     async_active  10.1.0.32              1000           1000            10  true                            1
20.0.201.0   default      default  Up       async_active  10.1.0.32              1000           1000            10  true                            2
admin@********:~$ redis-cli -n 6 hgetall "VNET_ROUTE_TUNNEL_TABLE|Vnet_55|3.2.1.0/32"
1) "active_endpoints"
2) "10.1.1.32"
3) "state"
4) "active"

Set secondary down. Both sessions are down. Route is removed.

admin@********:~$ redis-cli -n 6 hset  "BFD_SESSION_TABLE|default|default|20.0.201.0" state Down
(integer) 0
admin@********:~$ show bfd sum 
Total number of BFD sessions: 2
Peer Addr    Interface    Vrf      State    Type          Local Addr      TX Interval    RX Interval    Multiplier  Multihop      Local Discriminator
-----------  -----------  -------  -------  ------------  ------------  -------------  -------------  ------------  ----------  ---------------------
20.0.200.0   default      default  Down     async_active  10.1.0.32              1000           1000            10  true                            1
20.0.201.0   default      default  Down     async_active  10.1.0.32              1000           1000            10  true                            2
admin@********:~$ redis-cli -n 6 hgetall "VNET_ROUTE_TUNNEL_TABLE|Vnet_55|3.2.1.0/32"
(empty array)

Set secondary up. Nexthop pointer to secondary.

admin@********:~$ show bfd sum 
Total number of BFD sessions: 2
Peer Addr    Interface    Vrf      State    Type          Local Addr      TX Interval    RX Interval    Multiplier  Multihop      Local Discriminator
-----------  -----------  -------  -------  ------------  ------------  -------------  -------------  ------------  ----------  ---------------------
20.0.200.0   default      default  Down     async_active  10.1.0.32              1000           1000            10  true                            1
20.0.201.0   default      default  Up       async_active  10.1.0.32              1000           1000            10  true                            2
admin@********:~$ redis-cli -n 6 hgetall "VNET_ROUTE_TUNNEL_TABLE|Vnet_55|3.2.1.0/32"
1) "active_endpoints"
2) "10.1.1.32"
3) "state"
4) "active"

Set primary up. Both up, nexthop pointed to primary.

admin@********:~$ redis-cli -n 6 hset  "BFD_SESSION_TABLE|default|default|20.0.200.0" state Up
(integer) 0
admin@********:~$ show bfd sum 
Total number of BFD sessions: 2
Peer Addr    Interface    Vrf      State    Type          Local Addr      TX Interval    RX Interval    Multiplier  Multihop      Local Discriminator
-----------  -----------  -------  -------  ------------  ------------  -------------  -------------  ------------  ----------  ---------------------
20.0.200.0   default      default  Up       async_active  10.1.0.32              1000           1000            10  true                            1
20.0.201.0   default      default  Up       async_active  10.1.0.32              1000           1000            10  true                            2

admin@********:~$ redis-cli -n 6 hgetall "VNET_ROUTE_TUNNEL_TABLE|Vnet_55|3.2.1.0/32"
1) "active_endpoints"
2) "20.0.200.0"
3) "state"
4) "active"

Details if related

mssonicbld · 2025-10-08T15:19:08Z

/azp run

azure-pipelines · 2025-10-08T15:19:19Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-10-08T16:22:51Z

/azp run

azure-pipelines · 2025-10-08T16:23:02Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-10-08T23:08:41Z

/azp run

azure-pipelines · 2025-10-08T23:08:50Z

Azure Pipelines successfully started running 1 pipeline(s).

tests/test_vnet.py

+        update_bfd_session_state(dvs, '9.1.0.3', 'Up')
+        update_bfd_session_state(dvs, '9.1.0.4', 'Up')
+        time.sleep(2)
+        route1 = vnet_obj.check_priority_vnet_ecmp_routes(dvs, vnet_name, ['9.1.0.1','9.1.0.2'], tunnel_name, route_ids=route1)


… custom bfd; add tx rx for custom bfd

mssonicbld · 2025-10-10T09:44:20Z

/azp run

azure-pipelines · 2025-10-10T09:44:31Z

Azure Pipelines successfully started running 1 pipeline(s).

… custom_bfd

mssonicbld · 2025-10-10T10:03:38Z

/azp run

azure-pipelines · 2025-10-10T10:03:47Z

Azure Pipelines successfully started running 1 pipeline(s).

orchagent/vnetorch.cpp

… custom_bfd

mssonicbld · 2025-10-20T19:17:18Z

/azp run

azure-pipelines · 2025-10-20T19:17:28Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-12-02T02:30:14Z

/azp run

azure-pipelines · 2025-12-02T02:30:24Z

Azure Pipelines successfully started running 1 pipeline(s).

zjswhhh · 2025-12-02T06:27:54Z

/azpw Azure.sonic-swss

mssonicbld · 2025-12-02T10:26:29Z

/azp run

azure-pipelines · 2025-12-02T10:26:39Z

Azure Pipelines successfully started running 1 pipeline(s).

… custom_bfd

mssonicbld · 2025-12-07T08:05:39Z

/azp run

azure-pipelines · 2025-12-07T08:05:50Z

Azure Pipelines successfully started running 1 pipeline(s).

zjswhhh · 2025-12-08T04:55:57Z

Hi @prsunny - can we force merge this? BuildAsan amd64 build failure seems to be a generic issue again, it's not related to my change.

mssonicbld · 2025-12-08T17:00:29Z

/azp run

azure-pipelines · 2025-12-08T17:00:39Z

Azure Pipelines successfully started running 1 pipeline(s).

…ary/secondary NHG switchover (sonic-net#3922) What I did Today vnetorch only supports: custom monitoring sessions with primary & secondary NHG. bfd monitoring with primary NHG only. To make HA work we need bfd monitoring to support primary+secondary. This PR is to introduce a new monitoring type custom_bfd for that purpose. It will: create bfd sessions for both primary & secondary endpoints listen on state db for bfd session state update ignore update call from bfdorch switch and update nhg based on bfd session state It's not an elegant design to introduce a new bfd monitoring type. But the intention is to not break current functionality of vnetorch. Refactor of the code will be needed in future. Signed-off-by: Kalash Nainwal <kalash@nexthop.ai>

…ary/secondary NHG switchover (sonic-net#3922) What I did Today vnetorch only supports: custom monitoring sessions with primary & secondary NHG. bfd monitoring with primary NHG only. To make HA work we need bfd monitoring to support primary+secondary. This PR is to introduce a new monitoring type custom_bfd for that purpose. It will: create bfd sessions for both primary & secondary endpoints listen on state db for bfd session state update ignore update call from bfdorch switch and update nhg based on bfd session state It's not an elegant design to introduce a new bfd monitoring type. But the intention is to not break current functionality of vnetorch. Refactor of the code will be needed in future.

…ary/secondary NHG switchover (sonic-net#3922) What I did Today vnetorch only supports: custom monitoring sessions with primary & secondary NHG. bfd monitoring with primary NHG only. To make HA work we need bfd monitoring to support primary+secondary. This PR is to introduce a new monitoring type custom_bfd for that purpose. It will: create bfd sessions for both primary & secondary endpoints listen on state db for bfd session state update ignore update call from bfdorch switch and update nhg based on bfd session state It's not an elegant design to introduce a new bfd monitoring type. But the intention is to not break current functionality of vnetorch. Refactor of the code will be needed in future. Signed-off-by: Jing Zhang <zhangjing@microsoft.com>

* [vnetorch] missing handling of rx and tx interval of monitoring session (#3878) What I did Adding rx_monitor_timer and tx_monitor_timer handling per HLD: https://github.com/sonic-net/SONiC/blob/master/doc/vxlan/Overlay%20ECMP%20ehancements.md sign-off: Jing Zhang zhangjing@microsoft.com Why I did it It's needed for SSW HA scenario as DPU side bfd is a software solution, interval must be set to a reasonable value. Signed-off-by: Jing Zhang <zhangjing@microsoft.com> * [ssw][ha] introducing `custom_bfd` and BfdMonitorOrch to support primary/secondary NHG switchover (#3922) What I did Today vnetorch only supports: custom monitoring sessions with primary & secondary NHG. bfd monitoring with primary NHG only. To make HA work we need bfd monitoring to support primary+secondary. This PR is to introduce a new monitoring type custom_bfd for that purpose. It will: create bfd sessions for both primary & secondary endpoints listen on state db for bfd session state update ignore update call from bfdorch switch and update nhg based on bfd session state It's not an elegant design to introduce a new bfd monitoring type. But the intention is to not break current functionality of vnetorch. Refactor of the code will be needed in future. Signed-off-by: Jing Zhang <zhangjing@microsoft.com> --------- Signed-off-by: Jing Zhang <zhangjing@microsoft.com>

…ary/secondary NHG switchover (sonic-net#3922) What I did Today vnetorch only supports: custom monitoring sessions with primary & secondary NHG. bfd monitoring with primary NHG only. To make HA work we need bfd monitoring to support primary+secondary. This PR is to introduce a new monitoring type custom_bfd for that purpose. It will: create bfd sessions for both primary & secondary endpoints listen on state db for bfd session state update ignore update call from bfdorch switch and update nhg based on bfd session state It's not an elegant design to introduce a new bfd monitoring type. But the intention is to not break current functionality of vnetorch. Refactor of the code will be needed in future.

…c-net#3922) (sonic-net#182) **Cherry-pick conflict. Creating PR manually.** Picking commits below from public master branch: ``` b88a5c2 [ssw][ha] introducing `custom_bfd` and BfdMonitorOrch to support primary/secondary NHG switchover (sonic-net#3922) 8541200 [vnetorch] missing handling of rx and tx interval of monitoring session (sonic-net#3878) ```

…ary/secondary NHG switchover (sonic-net#3922) What I did Today vnetorch only supports: custom monitoring sessions with primary & secondary NHG. bfd monitoring with primary NHG only. To make HA work we need bfd monitoring to support primary+secondary. This PR is to introduce a new monitoring type custom_bfd for that purpose. It will: create bfd sessions for both primary & secondary endpoints listen on state db for bfd session state update ignore update call from bfdorch switch and update nhg based on bfd session state It's not an elegant design to introduce a new bfd monitoring type. But the intention is to not break current functionality of vnetorch. Refactor of the code will be needed in future. Signed-off-by: Baorong Liu <96146196+baorliu@users.noreply.github.com>

zjswhhh added 5 commits October 8, 2025 00:19

..

b953e2f

create monitor session

0d3e494

remove session

5032db7

doroutetask

d7b0499

minor

ade28ed

wrong key index

77f7f84

zjswhhh added 3 commits October 8, 2025 19:16

temp ut

796f062

rx timer change

0afdfee

define bfd statet table request

cce0425

github-advanced-security bot found potential problems Oct 8, 2025

View reviewed changes

fix race condition at bfd creating time; skip updateVnetTun when it's…

3183fd2

… custom bfd; add tx rx for custom bfd

Merge branch 'master' of https://github.com/sonic-net/sonic-swss into…

a33b526

… custom_bfd

zjswhhh marked this pull request as ready for review October 10, 2025 10:03

zjswhhh requested a review from prsunny as a code owner October 10, 2025 10:03

zjswhhh commented Oct 10, 2025

View reviewed changes

orchagent/vnetorch.cpp Outdated Show resolved Hide resolved

remove hard-coded monitoring type

57a625a

Copilot AI review requested due to automatic review settings October 20, 2025 19:17

Merge branch 'master' of https://github.com/sonic-net/sonic-swss into…

be67a58

… custom_bfd

mssonicbld added the Cherry Pick Conflict_msft-202506 label Dec 2, 2025

to trigger azp

03eed9e

zjswhhh added 2 commits December 7, 2025 08:05

fix UTs

3c648c0

Merge branch 'master' of https://github.com/sonic-net/sonic-swss into…

00e587a

… custom_bfd

zjswhhh added the Request for 202511 Branch label Dec 7, 2025

mssonicbld added the Cherry Pick Conflict_202511 label Dec 7, 2025

Merge branch 'master' into custom_bfd

8dabaff

prsunny merged commit b88a5c2 into sonic-net:master Dec 9, 2025
16 checks passed

prsunny added the Approved for 202511 Branch label Dec 9, 2025

prabhataravind removed Request for msft-202506 Branch Cherry Pick Conflict_msft-202506 labels Dec 9, 2025

Conversation

zjswhhh commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mssonicbld commented Oct 8, 2025

Uh oh!

azure-pipelines bot commented Oct 8, 2025

Uh oh!

mssonicbld commented Oct 8, 2025

Uh oh!

azure-pipelines bot commented Oct 8, 2025

Uh oh!

mssonicbld commented Oct 8, 2025

Uh oh!

azure-pipelines bot commented Oct 8, 2025

Uh oh!

Check notice

mssonicbld commented Oct 10, 2025

Uh oh!

azure-pipelines bot commented Oct 10, 2025

Uh oh!

mssonicbld commented Oct 10, 2025

Uh oh!

azure-pipelines bot commented Oct 10, 2025

Uh oh!

Uh oh!

mssonicbld commented Oct 20, 2025

Uh oh!

azure-pipelines bot commented Oct 20, 2025

Uh oh!

mssonicbld commented Dec 2, 2025

Uh oh!

azure-pipelines bot commented Dec 2, 2025

Uh oh!

zjswhhh commented Dec 2, 2025

Uh oh!

mssonicbld commented Dec 2, 2025

Uh oh!

azure-pipelines bot commented Dec 2, 2025

Uh oh!

mssonicbld commented Dec 7, 2025

Uh oh!

azure-pipelines bot commented Dec 7, 2025

Uh oh!

zjswhhh commented Dec 8, 2025

Uh oh!

mssonicbld commented Dec 8, 2025

Uh oh!

azure-pipelines bot commented Dec 8, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

zjswhhh commented Oct 8, 2025 •

edited

Loading