Getting L3 EVPN route-target based leaking working

[jcaamano]

Hi

I am trying to get L3 EVPN route-target based leaking working. My problem is that I do see the routes leaked in bgp but it is not resulting in zebra applying a working host configuration.

I have this containerlab.

There are four hosts: HOST1, HOST3 connected to leaf1, HOST2 and HOST4 connected to leaf2; HOST1,HOST2 are in L3 EVPN red/100; HOST3, HOST4 in blue/101. Initially, reachability across hosts works within their respective EVPNs.

Then, as an exercise to learn how route-target based leaking is configured and how it works, I want HOST4 to be able to reach to HOST1. For that I add route-target import 64512:100 to blue in leaf2.

It doesn't work. How it doesn't work depends on the FRR version.

Type-5 routes looks fine overall

❯ docker exec -ti clab-evpnl3-leaf2 vtysh -c "show bgp l2vpn evpn"
BGP table version is 1, local router ID is 100.64.0.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]:[Frag-id]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 192.168.11.1:2
 *> [5]:[0]:[24]:[192.168.11.0]
                    100.64.0.1                             0 64612 64512 ?
                    RT:64512:100 ET:8 Rmac:aa:bb:cc:00:00:64
Route Distinguisher: 192.168.12.1:2
 *> [5]:[0]:[24]:[192.168.12.0]
                    100.64.0.2               0         32768 ?
                    ET:8 RT:64512:100 Rmac:aa:bb:cc:00:00:65
Route Distinguisher: 192.168.13.1:3
 *> [5]:[0]:[24]:[192.168.13.0]
                    100.64.0.1                             0 64612 64512 ?
                    RT:64512:101 ET:8 Rmac:aa:bb:cc:00:01:64
Route Distinguisher: 192.168.14.1:3
 *> [5]:[0]:[24]:[192.168.14.0]
                    100.64.0.2               0         32768 ?
                    ET:8 RT:64512:101 Rmac:aa:bb:cc:00:01:65

Displayed 4 out of 4 total prefixes

---

For FRR v8, the remote mac for blue vni changes to an invalid value, from the gateway mac of blue network to the gateway mac of red network, so that blue traffic doesn't work anymore.
From:

❯ docker exec -ti clab-evpnl3-leaf2 vtysh -c "show evpn rmac vni 101"
Number of Remote RMACs known for this VNI: 1
MAC               Remote VTEP          
aa:bb:cc:00:01:64 100.64.0.1   

To:

❯ docker exec -ti clab-evpnl3-leaf2 vtysh -c "show evpn rmac vni 101"
Number of Remote RMACs known for this VNI: 1
MAC               Remote VTEP          
aa:bb:cc:00:00:64 100.64.0.1   

❯ docker exec -ti clab-evpnl3-leaf2 vtysh -c "show ip route vrf blue"
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF blue:
B>* 192.168.11.0/24 [20/0] via 100.64.0.1, br101 onlink, weight 1, 00:00:35
B>* 192.168.13.0/24 [20/0] via 100.64.0.1, br101 onlink, weight 1, 00:00:35
C>* 192.168.14.0/24 is directly connected, eth3, 00:00:37

---

For FRR v9+, the route is inactive and never installed because the labeled VNI for the leaked route does not match the VNI of the vxlan interface on the blue bridge.

❯ docker exec -ti clab-evpnl3-leaf2 vtysh -c "show ip route vrf blue"
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF blue:
B   192.168.11.0/24 [20/0] via 100.64.0.1, br101 inactive onlink, label 100, weight 1, 00:01:02
B>* 192.168.13.0/24 [20/0] via 100.64.0.1, br101 onlink, weight 1, 00:01:02
C>* 192.168.14.0/24 is directly connected, eth3, weight 1, 00:01:04
K * 192.168.14.0/24 [0/0] is directly connected, eth3 (vrf default), weight 1, 00:01:04
L>* 192.168.14.1/32 is directly connected, eth3, weight 1, 00:01:04

❯ docker exec -ti clab-evpnl3-leaf2 vtysh -c "show evpn rmac vni 101"
Number of Remote RMACs known for this VNI: 2
MAC               Remote VTEP          
aa:bb:cc:00:01:64 100.64.0.1           
aa:bb:cc:00:00:64 100.64.0.1   

---

I understand why this all is happening and it is clear that I am either missing something important and I don't know what; or that what I am trying to do can't be done with FRR.

I could not find any reference documentation, article or lab showcasing how route-target based leaking works with FRR EVPN and zebra. Mentions I see in EVPN documentation and articles I see of this working with other vendors make it look straight forward but guess it is not. I searched here and there are various issues that reached no conclusion. The only one that got it working is #14259 but I am not sure if that is the only way (I am not particularly interested in D-VNI or single vxlan devices) )or if there is a more canonical alternative than that.

Any help appreciated, thanks.

[jcaamano]

I managed to get the configuration to work with FRR v8. The trick was to ensure all bridges in a node had the same MAC. Once the mac issue was solved I proceeded to complete the configuration making sure cross leaking happened both in evpn and unicast address families. I have updated the lab to reflect that.

However, same configuration does not work in FRR v9+:

❯ docker exec -ti clab-evpnl3-leaf1 vtysh -c "show ip route vrf red"
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

IPv4 unicast VRF red:
C>* 192.168.11.0/24 is directly connected, eth2, weight 1, 00:05:40
L>* 192.168.11.1/32 is directly connected, eth2, weight 1, 00:05:40
B   192.168.12.0/24 [20/0] via 100.64.0.2, br100 inactive onlink, label 100, weight 1, 00:05:38
                           via 100.64.0.2, br100 inactive onlink, label 101, weight 1, 00:05:38
B>* 192.168.13.0/24 [20/0] is directly connected, blue (vrf blue), weight 1, 00:05:40
B   192.168.14.0/24 [20/0] via 100.64.0.2, br100 inactive onlink, label 100, weight 1, 00:05:38
                           via 100.64.0.2, br100 inactive onlink, label 101, weight 1, 00:05:38

From logs:

2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br100 D-VNI but route's vrf red(3) doesn't use SVD
2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br100 D-VNI but route's vrf red(3) doesn't use SVD
2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br100 D-VNI but route's vrf red(3) doesn't use SVD
2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br100 D-VNI but route's vrf red(3) doesn't use SVD
2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br101 D-VNI but route's vrf blue(6) doesn't use SVD
2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br101 D-VNI but route's vrf blue(6) doesn't use SVD
2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br101 D-VNI but route's vrf blue(6) doesn't use SVD
2025/10/01 16:03:37 ZEBRA: [GR56E-2Z1TY] nexthop 100.64.0.2, via br101 D-VNI but route's vrf blue(6) doesn't use SVD

I am not sure if to have this setup working we are expected to use single vxlan devices. In #14259 @taspelund mentions trying to install a route via the VNI in the route rather than via the L3VNI locally configured for the VRF as if we had a choice however I am not sure how we can force via the L3VNI locally configured for the VRF through configuration..