Skip to content

[action] [PR:23970] [arp_update]Resolve neighbors from config_db#24247

Merged
mssonicbld merged 1 commit intosonic-net:202505from
mssonicbld:cherry/202505/23970
Oct 10, 2025
Merged

[action] [PR:23970] [arp_update]Resolve neighbors from config_db#24247
mssonicbld merged 1 commit intosonic-net:202505from
mssonicbld:cherry/202505/23970

Conversation

@mssonicbld
Copy link
Collaborator

What changed

  • This change is being implemented in arp_update script to ensure neighbor resolution works properly after firmware upgrades and system repaves.
  • This change was originally developed and validated on 202205, 202211 images ([master] Resolve neighbors from config_db #15006), and is now being backported to 202411 and newer versions to maintain consistent neighbor resolution across images of all sonic versions.
  • The arp_update script now defines kernel neighbors (KERNEIGH4 and KERNEIGH6) based on different device subtype to properly handle DualToR.

Why I did it

After firmware upgrades/repaves, devices will experience neighbor resolution issues because kernel neighbor table can be empty/missing entries, and hence traffic going to certain neighbors will drop.

What is being fixed

  • In Non-DualToRs, FAILED/INCOMPLETE neighbors are excluded because this status represents connection issues.
  • In DualToRs, servers are connected to two ToR switches but only one path is active at a time. When a neighbor is reachable through the peer ToR switch, the local ToR switch will have FAILED/INCOMPLETE neighbor entries, which is an expected behavior.
    • The original code excluded FAILED/INCOMPLETE neighbors for all device types, which cause issues on DualToR devices: Neighbors that should be reachable via the peer switch but are FAILED in kernel wouldn't be detected as mismatches.
    • With the fix (post_upgrade), the standby ToR will include FAILED/INCOMPLETE neighbors in mismatch checking and will be included in synchronization processing since the script can detect the mismatch between kernel FAILED state and APPL_DB entries.

Example

# Immediately after system repave on DualToR standby switch
$ sonic-db-cli APPL_DB keys NEIGH_TABLE:Vlan100:*
NEIGH_TABLE:Vlan100:192.168.1.100
NEIGH_TABLE:Vlan100:192.168.1.101
NEIGH_TABLE:Vlan100:192.168.1.102

# Kernel starts with empty/failed entries
$ ip -4 neigh show | grep Vlan100
192.168.1.100 dev Vlan100 FAILED
192.168.1.101 dev Vlan100 FAILED
192.168.1.102 dev Vlan100 FAILED

# With enhanced arp_update script:
# 1. Includes FAILED entries in mismatch detection
# 2. Compares with APPL_DB entries
# 3. Triggers appropriate resolution (ping/tunnel route setup)
# 4. Results in proper neighbor state restoration

# Final state after arp_update processing:
$ ip -4 neigh show | grep Vlan100
192.168.1.100 dev Vlan100 lladdr 00:00:00:00:00:00 PERMANENT  # Zero MAC for peer-reachable
192.168.1.101 dev Vlan100 lladdr aa:bb:cc:dd:ee:ff REACHABLE  # Direct reachable
192.168.1.102 dev Vlan100 lladdr 00:00:00:00:00:00 PERMANENT  # Zero MAC for peer-reachable
Work item tracking
  • Microsoft ADO (number only):

How to verify it

Which release branch to backport (provide reason below if selected)

  • 202205
  • 202211
  • 202305
  • 202311
  • 202405
  • 202411
  • 202505

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

<!--
     Please make sure you've read and understood our contributing guidelines:
     https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md

     ** Make sure all your commits include a signature generated with `git commit -s` **

     If this is a bug fix, make sure your description includes "fixes #xxxx", or
     "closes #xxxx" or "resolves #xxxx"

     Please provide the following information:
-->
#### What changed
- This change is being implemented in `arp_update` script to ensure neighbor resolution works properly after firmware upgrades and system repaves.
- This change was originally developed and validated on 202205, 202211 images (sonic-net#15006), and is now being backported to 202411 and newer versions to maintain consistent neighbor resolution across images of all sonic versions.
- The `arp_update` script now defines kernel neighbors (`KERNEIGH4` and `KERNEIGH6`) based on different device subtype to properly handle DualToR.

#### Why I did it
After firmware upgrades/repaves, devices will experience neighbor resolution issues because kernel neighbor table can be empty/missing entries, and hence traffic going to certain neighbors will drop.

#### What is being fixed
- In Non-DualToRs, `FAILED/INCOMPLETE` neighbors are excluded because this status represents connection issues.
- In DualToRs, servers are connected to two ToR switches but only one path is active at a time. When a neighbor is reachable through the peer ToR switch, the local ToR switch will have FAILED/INCOMPLETE neighbor entries, which is an expected behavior.
    - The original code excluded `FAILED/INCOMPLETE` neighbors for all device types, which cause issues on DualToR devices: Neighbors that should be reachable via the peer switch but are `FAILED` in kernel wouldn't be detected as mismatches.
    - With the fix (post_upgrade), the standby ToR will include `FAILED/INCOMPLETE` neighbors in mismatch checking and will be included in synchronization processing since the script can detect the mismatch between kernel `FAILED` state and APPL_DB entries.

#### Example
```
# Immediately after system repave on DualToR standby switch
$ sonic-db-cli APPL_DB keys NEIGH_TABLE:Vlan100:*
NEIGH_TABLE:Vlan100:192.168.1.100
NEIGH_TABLE:Vlan100:192.168.1.101
NEIGH_TABLE:Vlan100:192.168.1.102

# Kernel starts with empty/failed entries
$ ip -4 neigh show | grep Vlan100
192.168.1.100 dev Vlan100 FAILED
192.168.1.101 dev Vlan100 FAILED
192.168.1.102 dev Vlan100 FAILED

# With enhanced arp_update script:
# 1. Includes FAILED entries in mismatch detection
# 2. Compares with APPL_DB entries
# 3. Triggers appropriate resolution (ping/tunnel route setup)
# 4. Results in proper neighbor state restoration

# Final state after arp_update processing:
$ ip -4 neigh show | grep Vlan100
192.168.1.100 dev Vlan100 lladdr 00:00:00:00:00:00 PERMANENT  # Zero MAC for peer-reachable
192.168.1.101 dev Vlan100 lladdr aa:bb:cc:dd:ee:ff REACHABLE  # Direct reachable
192.168.1.102 dev Vlan100 lladdr 00:00:00:00:00:00 PERMANENT  # Zero MAC for peer-reachable
```

##### Work item tracking
- Microsoft ADO **(number only)**:

#### How to verify it

<!--
If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012.
-->

#### Which release branch to backport (provide reason below if selected)

<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->

- [ ] 202205
- [ ] 202211
- [x] 202305
- [x] 202311
- [x] 202405
- [x] 202411
- [x] 202505

#### Tested branch (Please provide the tested image version)

<!--
- Please provide tested image version
- e.g.
- [x] 20201231.100
-->

- [ ] <!-- image version 1 -->
- [ ] <!-- image version 2 -->

#### Description for the changelog
<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->

<!--
 Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
-->

#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md
-->

#### A picture of a cute animal (not mandatory but encouraged)
@mssonicbld
Copy link
Collaborator Author

Original PR: #23970

@mssonicbld
Copy link
Collaborator Author

/azp run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld mssonicbld merged commit feb9a1a into sonic-net:202505 Oct 10, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant