FRR marks BGP route as invalid due to allegedly "inaccessible" next-hop #18300

g00g1 · 2025-03-03T13:31:21Z

g00g1
Mar 3, 2025

Simplified, I have the following FRR configuration:

router bgp 64512
 neighbor fe80::28f:cff:fe2b:5457 remote-as 65534
 neighbor fe80::28f:cff:fe2b:5457 interface 3543906233
 neighbor fe80::28f:cff:fe2b:5457 passive
 neighbor fe80::28f:cff:fe2b:5457 local-role provider
 neighbor fe80::28f:cff:fe2b:5457 update-source 3543906233
 neighbor fe80::28f:cff:fe2b:5457 capability extended-nexthop
 no neighbor fe80::28f:cff:fe2b:5457 capability fqdn
 !
 address-family ipv6 unicast
  redistribute kernel
  neighbor fe80::28f:cff:fe2b:5457 activate
  neighbor fe80::28f:cff:fe2b:5457 next-hop-self
  neighbor fe80::28f:cff:fe2b:5457 default-originate
  neighbor fe80::28f:cff:fe2b:5457 prefix-list pl-v6-generic-any in
  neighbor fe80::28f:cff:fe2b:5457 prefix-list pl-v6-generic-defroute out
  neighbor fe80::28f:cff:fe2b:5457 route-map rm-bgp-customer-in in
 exit-address-family
exit
!
route-map rm-bgp-customer-in permit 30
 match as-path al-cust-as65534
 set community 64512:3100 additive
 set local-preference 300
exit

BGP peer fe80::28f:cff:fe2b:5457 has global IPv6 address 2001:db8:fe00:5b::a that is routed to a next-hop fe80::28f:cff:fe2b:5457 via interface 3543906233, however FRR struggles to install route for received announcement of 2001:db8:ae16::/48:

# show bgp ipv6 2001:db8:ae16::/48
BGP routing table entry for 2001:db8:ae16::/48, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  65534
    2001:db8:fe00:5b::a (inaccessible, import-check enabled) from fe80::28f:cff:fe2b:5457 (x.x.x.x)
    (fe80::28f:cff:fe2b:5457) (used)
      Origin IGP, localpref 300, invalid, external
      Community: 64512:3100
      Last update: Mon Mar  3 13:17:01 2025

# show bgp nexthop detail
 2001:db8:fe00:5b::a overlay index unresolved [IGP metric 0], #paths 1
  Last update: Mon Mar  3 11:25:13 2025
  Paths:
    2/1 2001:db8:ae16::/48 flags 0x420
 fe80::28f:cff:fe2b:5457 valid [IGP metric 0], #paths 0, peer fe80::28f:cff:fe2b:5457
  Resolved prefix fe80::/64
  if 3951123873
  Last update: Mon Mar  3 11:54:30 2025
  Paths:

Is it expected behavior? Any suggestions how can I fix it?

ton31337 · 2025-03-03T13:36:00Z

ton31337
Mar 3, 2025
Collaborator

Which version?

1 reply

g00g1 Mar 3, 2025
Author

# sh version
FRRouting 10.2.1 (REDACTED) on Linux(5.14.0-427.22.1.el9_4.x86_64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-static' '--disable-werror' '--enable-multipath=256' '--enable-vtysh' '--enable-ospfclient' '--enable-ospfapi' '--enable-rtadv' '--enable-ldpd' '--enable-pimd' '--enable-pim6d' '--enable-pbrd' '--enable-nhrpd' '--enable-eigrpd' '--enable-babeld' '--enable-vrrpd' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-fpm' '--enable-watchfrr' '--disable-bgp-vnc' '--enable-isisd' '--enable-rpki' '--enable-bfdd' '--enable-pathd' '--disable-grpc' '--enable-snmp' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig' 'CC=gcc' 'CXX=g++' 'LT_SYS_LIBRARY_PATH=/usr/lib64:'

Still relevant for 10.3:

FRRouting 10.3.1 (REDACTED) on Linux(5.14.0-427.40.1.el9_4.x86_64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-static' '--disable-werror' '--enable-multipath=256' '--enable-vtysh' '--enable-ospfclient' '--enable-ospfapi' '--enable-rtadv' '--enable-ldpd' '--enable-pimd' '--enable-pim6d' '--enable-pbrd' '--enable-nhrpd' '--enable-eigrpd' '--enable-babeld' '--enable-vrrpd' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-fpm' '--enable-watchfrr' '--disable-bgp-vnc' '--enable-isisd' '--enable-doc' '--enable-rpki' '--enable-bfdd' '--enable-pathd' '--disable-grpc' '--enable-snmp' '--disable-zeromq' '--enable-pcre2posix' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig' 'CC=gcc' 'CXX=g++' 'LT_SYS_LIBRARY_PATH=/usr/lib64:'

ton31337 · 2025-03-03T13:41:55Z

ton31337
Mar 3, 2025
Collaborator

Could you enable debug bgp updates and show the logs? It should say why the route is marked as invalid.

1 reply

g00g1 Mar 3, 2025
Author

My bad, was looking to wrong SSH session, however debug logs still don't disclose much:

[XXWBM-V772F] fe80::28f:cff:fe2b:5457(Unknown) rcvd UPDATE w/ attr: , origin i, mp_nexthop 2001:db8:fe00:5b::a(fe80::28f:cff:fe2b:5457), path 65534
[YCKEM-GB33T] fe80::28f:cff:fe2b:5457(Unknown) rcvd 2001:db8:ae16::/48 IPv6 unicast
[M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from fe80::28f:cff:fe2b:5457

ton31337 · 2025-03-03T14:26:28Z

ton31337
Mar 3, 2025
Collaborator

Could you also show:

show ipv6 bgp nexthop
show ipv6 nht

And then enable debug bgp nht additionally (+ logs).

3 replies

g00g1 Mar 3, 2025
Author

# show ipv6 bgp nexthop
% Unknown command: show ipv6 bgp nexthop
# show ipv6 nht
VRF default:
 Resolve via default: on
2001:db8:fe00:5b::a(Connected)
 unresolved(Connected)
 Client list: bgp(fd 33)
fe80::28f:cff:fe2b:5457(Connected)
 resolved via connected, prefix fe80::/64
 is directly connected, 3951123873 (vrf default), weight 1
 Client list: bgp(fd 33)

ton31337 Mar 3, 2025
Collaborator

Sorry, show bgp nexthop

g00g1 Mar 3, 2025
Author

debug bgp nht:

[ZBY0C-GC5CK] bgp_unlink_nexthop_check: freeing bnc fe80::28f:cff:fe2b:5457/128(520)(0)(VRF default)
[VWNP2-DNMFV] Found existing bnc fe80::28f:cff:fe2b:5457/128(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x562af6c82680, resolved prefix fe80::/64
[VWNP2-DNMFV] Found existing bnc fe80::28f:cff:fe2b:5457/128(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x562af6c82680, resolved prefix fe80::/64
[VWNP2-DNMFV] Found existing bnc fe80::28f:cff:fe2b:5457/128(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x562af6c82680, resolved prefix fe80::/64
[W3X54-30BTE] Allocated bnc fe80::28f:cff:fe2b:5457/128(520)(0)(VRF default) peer 0x0
[M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from fe80::28f:cff:fe2b:5457 in vrf default

It looks strange that ifindex is 0, in ip-link it is 520:

520: 3543906233: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN mode DEFAULT group default qlen 1000

# show bgp nexthop
Current BGP nexthop cache:
 2001:db8:fe00:5b::a overlay index unresolved [IGP metric 0], #paths 0
  Last update: Mon Mar  3 11:25:13 2025
 fe80::28f:cff:fe2b:5457 valid [IGP metric 0], #paths 0, peer fe80::28f:cff:fe2b:5457
  Resolved prefix fe80::/64
  if 3951123873
  Last update: Mon Mar  3 14:19:21 2025
 fe80::7007:46ff:fe25:4f48 valid [IGP metric 0], #paths 1
  Resolved prefix fe80::/64
  if 3951123873
  Last update: Sun Mar  2 00:13:00 2025
 fe80::28f:cff:fe2b:5457 overlay index unresolved [IGP metric 0], #paths 1
  Last update: Thu Oct 17 11:41:06 2024

g00g1 · 2025-03-03T15:06:32Z

g00g1
Mar 3, 2025
Author

Another interesting observation:

# sh bgp ipv6 2001:db8:ae16::/48
BGP routing table entry for 2001:db8:ae16::/48, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  65534
    2001:db8:fe00:5b::a (inaccessible, import-check enabled) from fe80::28f:cff:fe2b:5457 (x.x.x.x)
    (fe80::28f:cff:fe2b:5457) (prefer-global)
      Origin IGP, localpref 300, invalid, external
      Community: 64512:3100
      Last update: Mon Mar  3 15:02:52 2025
# sh ipv6 route 2001:db8:fe00:5b::a
Routing entry for 2001:db8:fe00:5b::a/128
  Known via "kernel", distance 0, metric 1024, best
  Last update 02:48:13 ago
  * directly connected, 3543906233, weight 1

But debug bgp nht says:

[VWNP2-DNMFV] Found existing bnc fe80::28f:cff:fe2b:5457/128(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x562af6c82680, resolved prefix fe80::/64
[VWNP2-DNMFV] Found existing bnc fe80::28f:cff:fe2b:5457/128(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x562af6c82680, resolved prefix fe80::/64
[VWNP2-DNMFV] Found existing bnc fe80::28f:cff:fe2b:5457/128(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x562af6c82680, resolved prefix fe80::/64
[VWNP2-DNMFV] Found existing bnc 2001:db8:fe00:5b::a/128(0)(VRF default) flags 0x8e ifindex 0 #paths 0 peer 0x562a8aa398f0, resolved prefix UNK prefix
[M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from fe80::28f:cff:fe2b:5457 in vrf default

0 replies

g00g1 · 2025-09-17T09:22:38Z

g00g1
Sep 17, 2025
Author

Same problem but for IPv4:

# sh ip route vrf vps 10.10.10.95
Routing entry for 10.10.10.95/32
  Known via "kernel", distance 0, metric 0, vrf vps, best
  Last update 00:34:46 ago
  * directly connected, 1714570827, weight 1

# sh bgp vrf vps ipv4 10.40.100.0/24
BGP routing table entry for 10.40.100.0/24, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  2xxxx1
    10.10.10.95 (inaccessible, import-check enabled) from 10.10.10.95 (10.10.10.95)
      Origin IGP, localpref 300, invalid, external
      Last update: Wed Sep 17 11:01:40 2025
# show ip bgp vrf vps nexthop 10.10.10.95
 10.10.10.95 invalid, #paths 2, peer 10.10.10.95
  Must be Connected
  Last update: Tue Sep 16 12:25:34 2025
  Paths:
    1/1 10.40.100.0/24 VRF vps flags 0x400
    1/1 10.40.122.0/24 VRF vps flags 0x400

0 replies

FRR marks BGP route as invalid due to allegedly "inaccessible" next-hop #18300

Uh oh!

g00g1 Mar 3, 2025

Replies: 5 comments · 5 replies

Uh oh!

ton31337 Mar 3, 2025 Collaborator

Uh oh!

Uh oh!

g00g1 Mar 3, 2025 Author

Uh oh!

ton31337 Mar 3, 2025 Collaborator

Uh oh!

g00g1 Mar 3, 2025 Author

Uh oh!

ton31337 Mar 3, 2025 Collaborator

Uh oh!

g00g1 Mar 3, 2025 Author

Uh oh!

ton31337 Mar 3, 2025 Collaborator

Uh oh!

g00g1 Mar 3, 2025 Author

Uh oh!

g00g1 Mar 3, 2025 Author

Uh oh!

Uh oh!

g00g1 Sep 17, 2025 Author

g00g1
Mar 3, 2025

Replies: 5 comments 5 replies

ton31337
Mar 3, 2025
Collaborator

g00g1 Mar 3, 2025
Author

ton31337
Mar 3, 2025
Collaborator

g00g1 Mar 3, 2025
Author

ton31337
Mar 3, 2025
Collaborator

g00g1 Mar 3, 2025
Author

ton31337 Mar 3, 2025
Collaborator

g00g1 Mar 3, 2025
Author

g00g1
Mar 3, 2025
Author

g00g1
Sep 17, 2025
Author