Segment Routing with IPv6(SRv6) update in Jun 23, 2021 brought in changes that keeps SRv6 APIs inline with MPLS. This update required SAI to be invoked with a single SIDList for SRv6 Next hop creation.
In a VPN application, the ingress node first identifies a flow to a given policy. To direct the packet, the ingress node then determines the end node, the VPN SID (a.k.a Overlay SID) to use and optionally the appropriate path in the underlay network to use.
The packet after the SR encapsulation combines both an underlay SID List as well as an overlay SID (that is used to build the VPN service). It is common to keep these two items disaggregated, as the underlay SID list is shared by many different flows; whereas the overlay SID is only used by the packets belong to that VPN service. This helps to scale the hardware, as those two objects are stored in different databases with different scale. Typically the number of VPN services is much higher.
The egress node upon receiving the packet identifies the service (VRF) using the destination IP which is the VPN SID (a.k.a overlay SID).
- SAI SID List object represents a path
- SRv6 SAI Tunnel object represents end node
SR Policy is represented by combination of <color, end node>. One or more paths could be created per color. Therefore a SR Policy is realized by combination of SAI SID List and SAI Tunnel objects.
Following changes are defined in this spec.
- Disaggregation of VPN SID from underlay SID List by use of tunnel map.
- Selection of underlay nexthop to use for a given SID list
- L3VPN only solution, without the need to use SID List.
- Allow prefix based derivation of VPN SID.
- Ability for multiple prefixes to same destination but different VPN SID to share same NH/NHG using Prefix aggregation id.
- Statistics collection on SID List
Figure 1: SRv6 VPN Topology.
In the above topology, Node 11 and Node 21/22 act in a Provider Edge (PE) Role. Overlay customer traffic is classified into a VRF and sent to a remote over the underlay core network. A policy is a combination of both color and end node. The end node on a specific policy can be reached via multiple underlay Segment Lists (SL) or tunnels.
There are 2 policies above, Orange and Green.
Remote network 10.0.0.0/8 on VRF A can be reached via two policies:
Policy 1 - NODE21, Orange
Policy 2 - NODE22, Orange
End node Node 21 and Node 22 are both reachable through two path's, SL1 and SL2.
Segment List SL1 - [31, 41, 51]
Segment List SL2 - [32, 42, 52]
NOS would need to create hierarchical ECMP groups For the Policy.
Level 1 - ECMP of Endpoints - NODE21, NODE22.
Level 2 - ECMP of Paths - SL1 and SL2.
NOS would flatten the above 2 list into 4 paths as below.
- NODE21, SL1
- NODE21, SL2
- NODE22, SL1
- NODE22, SL2
Remote network 12/8 on VRF A can be reached via policy 3.
Policy 3 - NODE21, Green
In this case, the NOS needs only needs to create a SID List(SL3) and create NH with End node 21.
A VPN SID is added to the SID list to the tenant or VRF. The VPN SID value is generated by the end node and propagated through IGP/EGP (such as BGP) or via CLI/management tools by a controller.
When traffic to 10/8 network is transported using the above sidlist, the VPN Sid value would be different for NODE21 and NODE22.
Traffic to NODE21, SL1 would have below header.
Outer IPv6 DA: FD00:201:31:41:51::
SRH Header:
FD00:201:XXX:FFF1:A/80 (VPN_SID_NODE21,ORANGE)
SAI IP tunnels specify the underlay VRF table to use to forward the encapsulated packet. The lookup in the underlay VRF table should result in a next hop to which the encapsulated packet is forwarded. The same behavior is used in SRv6 encapsulation as well.
In some deployment scenarios (typically brownfield deployment), it is desirable to direct the SRv6 encapsulated traffic to a specific next hop in underlay.
SAI API must provide a way for the NOS/Controller to program the SRv6 aware underlay next hop to use.
In current SAI SRv6 specification, the SRv6 NHs are configured using SRv6 Tunnel OID and a SID List OID. When multiple VPN SID's exist behind an end node, NOS would need to create several SID Lists, one per VPN SID. SID Lists with VPN SID would be {, VPN SID}. As a result, the number of SRv6 SID Lists (and NH's) required to support all endpoints with the VPN SID would be # VPN SID's * underlay SRv6 paths
Also any change in underlay SRv6 path would require NOS to update VPN SID's * SRv6 SID Lists.
Allow statistics collection per SID List (per path). This allows SONiC/NOS to collect statistics per policy.
L3VPN only use case is when SRv6 is used to provide SRv6 L3VPN solution while the underlay forwarding protocol are segregated, and hence the PE would not impose the entire SRv6 path. This allows deployment of SRv6 solution as the network is being upgraded to be SRv6 aware.
Figure 1: SRv6 L3VPN Topology.
Traffic to 12/8 network/VPN NODE21, would have below header.
Outer IPv6 DA: FD00:201:XXX:FFF1:A/80 (VPN_SID_NODE21,ORANGE)
Figure 1: SRv6 pipeline
Prefix Aggregation ID is a number that identifies a target on the overlay network, such as a VPN service. The route can be associated to a single SRv6 Next Hop (Endpoint) or a Next hop group of SRv6 NH's for the VPN. Prefix Aggregation Id allows SAI to derive VPN SID per SRv6 Endpoint (SRv6 Tunnel) when muliple SRv6 Endpoints are part of the VPN service.
The tunnel map entry table is programmed with the prefix aggregation id to VPN SID mapping.
L3VPN solution is achieved by allowing SIDLIST OID to be NULL or creating a SIDLIST OID with empty sidlist array. SIDList with empty sidlist array could be used for cases where the encapsulated packets need to be directed to a specific underlay NH.
Every SRv6 End node is represented by a SAI P2P Tunnel entry. All the VPN SIDs on this end node are specified using a tunnel map.
Tunnel Map entry table contains mapping of Prefix aggregation ID to VPN SID of the remote end node.
The diagram below shows the objects and relationship. The diagram only shows relevant fields in the different tables.
NOTE: Below object types have been combined to keep the diagram simple.
- Next hop group and member list
- Tunnel map and Tunnel map entry table
New objects are highlighed in RED.
Figure 1: SRv6 VPN Encap model.
- Create SAI tunnel for end node.
- Create a tunnel map per end node. A tunnel map entry can be used to map Prefix aggregation id (or VRF) to a VPN SID.
- Create a SAI Tunnel object for each end node (21,22 in above diagram). Attach the relevant tunnel map to it.
- Create underlay SID lists, similiar to existing capabilities. SID Lists are shared between all the VPN traffic that use same policy.
- Create SRv6 NH's, associating the underlay sidlists and the tunnel object.
Tunnel map type = SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID
Tunnel map entries = Database of mappings of prefix aggregation id to VPN SID
{ SAI_TUNNEL_MAP_ENTRY_ATTR_PREFIX_AGG_ID_KEY : SAI_TUNNEL_MAP_ENTRY_ATTR_SRV6_VPN_SID_VALUE }
Prefix Aggregation ID identifies a policy at the remote peer.
typedef enum _sai_route_entry_attr_t
{
// ...
// <snip>
// ...
/**
* @brief Prefix Aggregation ID
*
* Prefix Aggregation ID identifies specific policy at the remote target.
* Used in IPv6 Segment Routing VPN forwarding.
*
* @type sai_uint32_t
* @flags CREATE_AND_SET
* @default 0
*/
SAI_ROUTE_ENTRY_ATTR_PREFIX_AGG_ID,
} sai_route_entry_attr_t;Two new types of tunnel map's.
- Virtual Router ID (VRF) to VPN SID
- Prefix Aggregation ID to VPN SID
typedef enum _sai_tunnel_map_type_t
{
// ...
// <snip>
// ...
/** TUNNEL Map Virtual Router ID to SRv6 VPN SID */
SAI_TUNNEL_MAP_TYPE_VIRTUAL_ROUTER_ID_TO_VPN_SID = 0x0000000c,
/** TUNNEL Map Prefix Aggregation ID to SRv6 VPN SID */
SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID = 0x0000000d,
/** Custom range base value */
SAI_TUNNEL_MAP_TYPE_CUSTOM_RANGE_BASE = 0x10000000
} sai_tunnel_map_type_t;typedef enum _sai_tunnel_map_entry_attr_t
{
// ...
// <snip>
// ...
/**
* @brief IP Prefix key
*
* @type sai_uint32_t
* @flags MANDATORY_ON_CREATE | CREATE_ONLY
* @condition SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE == SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID
*/
SAI_TUNNEL_MAP_ENTRY_ATTR_PREFIX_AGG_ID_KEY = 0x00000010,
/**
* @brief SRv6 SID List ID value for VPN
*
* @type sai_object_id_t
* @flags MANDATORY_ON_CREATE | CREATE_ONLY
* @condition SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE == SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID or SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE == SAI_TUNNEL_MAP_TYPE_VIRTUAL_ROUTER_ID_TO_VPN_SID
*/
SAI_TUNNEL_MAP_ENTRY_ATTR_SRV6_VPN_SID_VALUE = 0x00000011,
// ...
// <snip>
// ...
} sai_tunnel_map_entry_attr_t;typedef enum _sai_srv6_sidlist_attr_t
{
// <...>
/**
* @brief Underlay Next hop to forward packets to this SID List
*
* @type sai_object_id_t
* @flags CREATE_AND_SET
* @objects SAI_OBJECT_TYPE_NEXT_HOP
* @allownull true
* @default SAI_NULL_OBJECT_ID
*/
SAI_SRV6_SIDLIST_ATTR_NEXT_HOP_ID,
// <...>
} sai_srv6_sidlist_attr_t;/**
* @brief SRv6 Sidlist counter IDs
*/
typedef enum _sai_srv6_sidlist_stat_t
{
/** Egress packet stat count */
SAI_SRV6_SIDLIST_STAT_OUT_PACKETS,
/** Egress byte stat count */
SAI_SRV6_SIDLIST_STAT_OUT_OCTETS,
} sai_srv6_sidlist_stat_t;/**
* @brief Get SRv6 Sidlist statistics counters. Deprecated for backward compatibility.
*
* @param[in] srv6_sidlist_id SRv6 Sidlist id
* @param[in] number_of_counters Number of counters in the array
* @param[in] counter_ids Specifies the array of counter ids
* @param[out] counters Array of resulting counter values.
*
* @return #SAI_STATUS_SUCCESS on success, failure status code on error
*/
typedef sai_status_t (*sai_get_srv6_sidlist_stats_fn)(
_In_ sai_object_id_t srv6_sidlist_id,
_In_ uint32_t number_of_counters,
_In_ const sai_stat_id_t *counter_ids,
_Out_ uint64_t *counters);
/**
* @brief Get SRv6 Sidlist statistics counters extended.
*
* @param[in] srv6_sidlist_id SRv6 Sidlist id
* @param[in] number_of_counters Number of counters in the array
* @param[in] counter_ids Specifies the array of counter ids
* @param[in] mode Statistics mode
* @param[out] counters Array of resulting counter values.
*
* @return #SAI_STATUS_SUCCESS on success, failure status code on error
*/
typedef sai_status_t (*sai_get_srv6_sidlist_stats_ext_fn)(
_In_ sai_object_id_t srv6_sidlist_id,
_In_ uint32_t number_of_counters,
_In_ const sai_stat_id_t *counter_ids,
_In_ sai_stats_mode_t mode,
_Out_ uint64_t *counters);
/**
* @brief Clear SRv6 Sidlist statistics counters.
*
* @param[in] srv6_sidlist_id SRv6 Sidlist id
* @param[in] number_of_counters Number of counters in the array
* @param[in] counter_ids Specifies the array of counter ids
*
* @return #SAI_STATUS_SUCCESS on success, failure status code on error
*/
typedef sai_status_t (*sai_clear_srv6_sidlist_stats_fn)(
_In_ sai_object_id_t srv6_sidlist_id,
_In_ uint32_t number_of_counters,
_In_ const sai_stat_id_t *counter_ids);typedef struct _sai_srv6_api_t
{
// <...>
sai_get_srv6_sidlist_stats_fn get_srv6_sidlist_stats;
sai_get_srv6_sidlist_stats_ext_fn get_srv6_sidlist_stats_ext;
sai_clear_srv6_sidlist_stats_fn clear_srv6_sidlist_stats;
// <...>
} sai_srv6_api_t;Programming with SRv6 VPN encap matching IP Prefix and VRF criteria
- Create tunnel with appropriate tunnel map type
- When the remote end is discovered.
- Create tunnel map entries
- When overlay route and VPN SIDs are discovered
- Create SRv6 NH's (and NH groups)
- When the underlay path to remote endpoint is discovered
- Create overlay route with SRv6 NH
- When overlay routes need to be installed.
// Node 11: fd00:201:a11::1
// Node 21: fd00:201:b21::1
// Create VPN to 10.0.0.0/8 on NODE21
// Underlay paths to NODE21:
// [31, 41, 51] = FD00:0201:31:E041:51::
// [32, 42, 52] = FD00:0201:32:E042:52::
// Setup weighted ECMP with the above 2 paths
// VPN Configuration
// VPN Sid: FD00:0201:b:FFF0:1234::
// 1. Following are already assumed to be setup.
// - Underlay RIF (loopback or whatever)
// - Underlay VRF and overlay VRF
// - IGP routes to reach 31 with required nexthops/next hop groups
//////// 2. New remote endpoint discovered [NODE21]
// Create a P2P Tunnel with SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID or SAI_TUNNEL_MAP_TYPE_VIRTUAL_ROUTER_ID_TO_VPN_SID type tunnel maps.
// 2.1 Create a tunnel map
tunnel_map_attrs[0].id = SAI_TUNNEL_MAP_ATTR_TYPE;
tunnel_map_attrs[0].value = SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID;
saistatus = sai_tunnel_api->create_tunnel_map(&node21_srv6_tunnel_encap_map, switch_id, 1, tunnel_map_attrs);
// 2.2 Create a SRv6 tunnel object for NODE21
tunnel_entry_attrs[0].id = SAI_TUNNEL_ATTR_TYPE;
tunnel_entry_attrs[0].value = SAI_TUNNEL_TYPE_SRV6;
tunnel_entry_attrs[1].id = SAI_TUNNEL_ATTR_ENCAP_SRC_IP;
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[1].value, "fd00:201:a11::1"); // SIP
tunnel_entry_attrs[2].id = SAI_TUNNEL_ATTR_UNDERLAY_INTERFACE; // Underlay
tunnel_entry_attrs[2].value.oid = underlay_rif;
tunnel_entry_attrs[3].id = SAI_TUNNEL_ATTR_ENCAP_MAPPERS; // Encap mapper
tunnel_entry_attrs[3].value.objlist.count = 1;
tunnel_entry_attrs[3].value.objlist.list[0] = node21_srv6_tunnel_encap_map;
tunnel_entry_attrs[4].id = SAI_TUNNEL_ATTR_PEER_MODE; // P2P mode
tunnel_entry_attrs[4].value = SAI_TUNNEL_PEER_MODE_P2P;
tunnel_entry_attrs[5].id = SAI_TUNNEL_ATTR_ENCAP_DST_IP; // Node 22
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[5].value, "fd00:201:b11::1");
saistatus = sai_tunnel_api->create_tunnel(&node_21_tunnel_id, switch_id, 6, tunnel_entry_attrs);
//////// 3. Setup Underlay tunnel to remote endpoint.
// Here there are 2 paths (sidlist_1 and sidlist_2) to NODE21.
// This is being setup as an ECMP with different weights
// 3.1 Create a Underlay SID list object (sidlist1), NODE21-SL1
sidlist_entry_attrs[0].id = SAI_SRV6_SIDLIST_ATTR_TYPE;
sidlist_entry_attrs[0].value.s32 = SAI_SRV6_SIDLIST_TYPE_ENCAPS_RED;
sidlist_entry_attrs[1].id = SAI_SRV6_SIDLIST_ATTR_SEGMENT_LIST;
sidlist_entry_attrs[1].value.objlist.count = 1;
CONVERT_STR_TO_IPV6(sidlist_entry_attrs[1].value.objlist.list[0], "FD00:0201:31:E041:51:");
saistatus = sai_srv6_api->create_srv6_sidlist(&sidlist_1, switch_id, 2, sidlist_entry_attrs);
// 3.2 Create a SRV6 nexthop object bound to the SID list object, NODE21_TNL_1
nexthop_entry_attrs[0].id = SAI_NEXTHOP_ATTR_TYPE;
nexthop_entry_attrs[0].value = SAI_NEXT_HOP_TYPE_SRV6_SIDLIST;
nexthop_entry_attrs[1].id = SAI_NEXTHOP_ATTR_TUNNEL_ID;
nexthop_entry_attrs[1].value.oid = node_21_tunnel_id;
nexthop_entry_attrs[2].id = SAI_NEXT_HOP_ATTR_SRV6_SIDLIST_ID;
nexthop_entry_attrs[2].value.oid = sidlist_1;
saistatus = sai_nexthop_api->create_next_hop(&srv6_nh_1, switch_id, 3, nexthop_entry_attrs);
// Repeat same steps to create srv6_nh_2, NODE21-SL2
// 3.1 Create a Underlay SID list object (sidlist2)
sidlist_entry_attrs[0].id = SAI_SRV6_SIDLIST_ATTR_TYPE;
sidlist_entry_attrs[0].value.s32 = SAI_SRV6_SIDLIST_TYPE_ENCAPS_RED;
sidlist_entry_attrs[1].id = SAI_SRV6_SIDLIST_ATTR_SEGMENT_LIST;
sidlist_entry_attrs[1].value.objlist.count = 1;
CONVERT_STR_TO_IPV6(sidlist_entry_attrs[1].value.objlist.list[0], "FD00:0201:32:E042:52:");
saistatus = sai_srv6_api->create_srv6_sidlist(&sidlist_2, switch_id, 2, sidlist_entry_attrs);
// 3.2 Create a SRV6 nexthop object bound to the SID list object, NODE21_TNL_2
nexthop_entry_attrs[0].id = SAI_NEXTHOP_ATTR_TYPE;
nexthop_entry_attrs[0].value = SAI_NEXT_HOP_TYPE_SRV6_SIDLIST;
nexthop_entry_attrs[1].id = SAI_NEXTHOP_ATTR_TUNNEL_ID;
nexthop_entry_attrs[1].value.oid = node_21_tunnel_id;
nexthop_entry_attrs[2].id = SAI_NEXT_HOP_ATTR_SRV6_SIDLIST_ID;
nexthop_entry_attrs[2].value.oid = sidlist_2;
saistatus = sai_nexthop_api->create_next_hop(&srv6_nh_2, switch_id, 3, nexthop_entry_attrs);
// 3.3 Create a ecmp group for multiple paths.. End Point ECMP Group -- NODE21-SL1 and NODE21-SL2
// NOTE: This is still default level (0) as it doesn't use another ECMP group as member.
nexthop_group_attrs[0].id = SAI_NEXT_HOP_GROUP_ATTR_TYPE;
nexthop_group_attrs[0].value.u32 = SAI_NEXT_HOP_GROUP_TYPE_ECMP;
saistatus = sai_nexthop_group_api->create_next_hop_group(&srv6_nh_grp, switch_id, 1, nexthop_group_attrs);
// 3.3.1 Create member with weight 3
nexthop_group_mbr_attrs[0].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_GROUP_ID;
nexthop_group_mbr_attrs[0].value = srv6_nh_grp;
nexthop_group_mbr_attrs[1].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_ID;
nexthop_group_mbr_attrs[1].value = srv6_nh_1;
nexthop_group_mbr_attrs[2].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_WEIGHT;
nexthop_group_mbr_attrs[2].value = 3;
saistatus = sai_nexthop_group_api->create_next_hop_group_member(&node21_nh_grp_mbr1, switch_id, 3, nexthop_group_mbr_attrs);
// 3.3.2 Create member with weight 1
nexthop_group_mbr_attrs[0].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_GROUP_ID;
nexthop_group_mbr_attrs[0].value = srv6_nh_grp;
nexthop_group_mbr_attrs[1].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_ID;
nexthop_group_mbr_attrs[1].value = srv6_nh_2;
nexthop_group_mbr_attrs[2].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_WEIGHT;
nexthop_group_mbr_attrs[2].value = 1;
saistatus = sai_nexthop_group_api->create_next_hop_group_member(&node21_nh_grp_mbr2, switch_id, 3, nexthop_group_mbr_attrs);
// (Omitted) Create additional NHG members to NODE22 via NODE22_TNL_1 and NODE22_TNL_2 to srv6_nh_grp
/////// 5. VPN SID with for a prefix discovered
// Create a new tunnel map entry for this SID.
// 5.1 Create a overlay VRF [overlay_vrf] (if it wasn't created already)
// Adding ports to VRF omitted here.
saistatus = sai_virtual_router_api->create_virtual_router(&overlay_vrf, switch_id, 0, virtual_router_attrs);
// 5.2 Create a VPN SID list object (vpn_sidlist_id) -- VPN-SID-1
sidlist_entry_attrs[0].id = SAI_SRV6_SIDLIST_ATTR_TYPE;
sidlist_entry_attrs[0].value.s32 = SAI_SRV6_SIDLIST_TYPE_ENCAPS_RED;
sidlist_entry_attrs[1].id = SAI_SRV6_SIDLIST_ATTR_SEGMENT_LIST;
sidlist_entry_attrs[1].value.objlist.count = 1;
CONVERT_STR_TO_IPV6(sidlist_entry_attrs[1].value.objlist.list[0], "FD00:0201:A20:FFF0:1234::");
saistatus = sai_srv6_api->create_srv6_sidlist(&vpn_sidlist_id, switch_id, 2, sidlist_entry_attrs);
// 5.3 Create a tunnel map entry with prefix aggregation id to SRV6 vpn_sidlist_id (Prefix-Agg-ID-1 : VPN_SID_1)
prefix_agg_id = 1;
tunnel_map_entry_attrs[0].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE;
tunnel_map_entry_attrs[0].value.u32 = SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID;
tunnel_map_entry_attrs[1].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP;
tunnel_map_entry_attrs[1].value.oid = node21_srv6_tunnel_encap_map;
tunnel_map_entry_attrs[2].id = SAI_TUNNEL_MAP_ENTRY_ATTR_PREFIX_AGG_ID_KEY;
tunnel_map_entry_attrs[2].value.u32 = prefix_agg_id; // Prefix Aggregation ID
tunnel_map_entry_attrs[3].id = SAI_TUNNEL_MAP_ENTRY_ATTR_SRV6_VPN_SID_VALUE;
tunnel_map_entry_attrs[3].value.oid = vpn_sidlist_id;
saistatus = sai_tunnel_api->create_tunnel_map_entry(&srv6_tunnel_map_entry, switch_id, 4, tunnel_map_entry_attrs);
// 6. Finally create (or update existing) overlay route entry using the nexthop
route_entry.switch_id = 0;
route_entry.vr_id = overlay_vrf;
route_entry.destination.addr_family = SAI_IP_ADDR_FAMILY_IPV4;
route_entry.destination.addr.ip4 = "10.0.0.0"; // {10.0.0.0/8 }
route_entry.destination.addr.mask = "255.0.0.0";
route_entry_attrs[0].id = SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID;
route_entry_attrs[0].value.oid = srv6_nh_grp;
route_entry_attrs[1].id = SAI_ROUTE_ENTRY_ATTR_PREFIX_AGG_ID;
route_entry_attrs[1].value.u32 = prefix_agg_id;
saistatus = sai_route_api->create_route(&route_entry, 2, route_entry_attrs);To address cases where there is no explicit SRv6 path to the remote endpoint, following programming example can be used.
The method follows similar steps as above, except that the underlay NH created has a NULL object id for SID-list.
// Create SRv6 VPN encap to 10.0.0.0/8 on NODE21
// Underlay paths to NODE21:
// Discovered via routing protocol and updated separately
// VPN Configuration
// VPN Sid: FD00:0201:A20:FFF0:1234::
// 1. Following are already assumed to be setup.
// - Underlay RIF (loopback or whatever)
// - Underlay VRF and overlay VRF
// - IGP Route/NH to VPN Sid in underlay
////// 2. Create SRv6 tunnel
// 2.1 Create a tunnel map
tunnel_map_attrs[0].id = SAI_TUNNEL_MAP_ATTR_TYPE;
tunnel_map_attrs[0].value = SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID;
saistatus = sai_tunnel_api->create_tunnel_map(&srv6_tunnel_encap_map, switch_id, 1, tunnel_map_attrs);
// 2.2 Create a P2P tunnel object to NODE21 with source IP used for SRv6 tunnels
tunnel_entry_attrs[0].id = SAI_TUNNEL_ATTR_TYPE;
tunnel_entry_attrs[0].value = SAI_TUNNEL_TYPE_SRV6;
tunnel_entry_attrs[1].id = SAI_TUNNEL_ATTR_ENCAP_SRC_IP;
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[1].value, "2001:db8:85a3::8a2e:370:9876");
tunnel_entry_attrs[2].id = SAI_TUNNEL_ATTR_UNDERLAY_INTERFACE;
tunnel_entry_attrs[2].value.oid = underlay_rif;
tunnel_entry_attrs[3].id = SAI_TUNNEL_ATTR_ENCAP_MAPPERS;
tunnel_entry_attrs[3].value.objlist.count = 1;
tunnel_entry_attrs[3].value.objlist.list[0] = srv6_tunnel_encap_map;
tunnel_entry_attrs[4].id = SAI_TUNNEL_ATTR_PEER_MODE;
tunnel_entry_attrs[4].value = SAI_TUNNEL_PEER_MODE_P2P;
tunnel_entry_attrs[5].id = SAI_TUNNEL_ATTR_ENCAP_DST_IP;
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[5].value, "2001:db8:85a3::8aff:1111:21");
saistatus = sai_tunnel_api->create_tunnel(&srv6_tunnel_id, switch_id, 6, tunnel_entry_attrs);
////// 3. Create Underlay SRv6 NH with null sidlist, NODE21 NH
// 3.1 Create a SRV6 nexthop object bound to the SID list object and H.Encaps.Red behavior
nexthop_entry_attrs[0].id = SAI_NEXTHOP_ATTR_TYPE;
nexthop_entry_attrs[0].value = SAI_NEXT_HOP_TYPE_SRV6_SIDLIST;
nexthop_entry_attrs[1].id = SAI_NEXTHOP_ATTR_TUNNEL_ID;
nexthop_entry_attrs[1].value.oid = srv6_tunnel_id;
nexthop_entry_attrs[2].id = SAI_NEXT_HOP_ATTR_SRV6_SIDLIST_ID;
nexthop_entry_attrs[2].value.oid = SAI_NULL_OBJECT_ID; // NULL SID-list
saistatus = sai_nexthop_api->create_next_hop(&srv6_nh, switch_id, 3, nexthop_entry_attrs);
// 3.2 Create a ecmp group for multiple paths to reach VPN SID-list, End point ECMP
nexthop_group_attrs[0].id = SAI_NEXT_HOP_GROUP_ATTR_TYPE;
nexthop_group_attrs[0].value.u32 = SAI_NEXT_HOP_GROUP_TYPE_ECMP;
saistatus = sai_nexthop_group_api->create_next_hop_group(&srv6_nh_grp, switch_id, 1, nexthop_group_attrs);
// 3.2.1 Create member with weight 1
nexthop_group_mbr_attrs[0].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_GROUP_ID;
nexthop_group_mbr_attrs[0].value = srv6_nh_grp;
nexthop_group_mbr_attrs[1].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_NEXT_HOP_ID;
nexthop_group_mbr_attrs[1].value = srv6_nh;
nexthop_group_mbr_attrs[2].id = SAI_NEXT_HOP_GROUP_MEMBER_ATTR_WEIGHT;
nexthop_group_mbr_attrs[2].value = 1;
saistatus = sai_nexthop_group_api->create_next_hop_group_member(&node21_nh_grp_mbr1, switch_id, 3, nexthop_group_mbr_attrs);
// (Omitted) Create additional NHG members to NODE22 to srv6_nh_grp
/////// 4. VPN SID with for a prefix discovered
// Create a new tunnel map entry for this SID.
// 4.1 Create a overlay VRF [overlay_vrf] (if it wasn't created already)
// Adding ports to VRF omitted here.
saistatus = sai_virtual_router_api->create_virtual_router(&overlay_vrf, switch_id, 0, virtual_router_attrs);
// 4.2 Create a VPN SID list object (vpn_sidlist_id) -- VPN-SID-1
sidlist_entry_attrs[0].id = SAI_SRV6_SIDLIST_ATTR_TYPE;
sidlist_entry_attrs[0].value.s32 = SAI_SRV6_SIDLIST_TYPE_ENCAPS_RED;
sidlist_entry_attrs[1].id = SAI_SRV6_SIDLIST_ATTR_SEGMENT_LIST;
sidlist_entry_attrs[1].value.objlist.count = 1;
CONVERT_STR_TO_IPV6(sidlist_entry_attrs[1].value.objlist.list[0], "FD00:0201:A20:FFF0:1234::");
saistatus = sai_srv6_api->create_srv6_sidlist(&vpn_sidlist_id, switch_id, 2, sidlist_entry_attrs);
// 4.3 Create a tunnel map entry with prefix aggregation id to SRV6 vpn_sidlist_id (Prefix-Agg-ID-1 : VPN_SID_1)
prefix_agg_id = 10;
tunnel_map_entry_attrs[0].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE;
tunnel_map_entry_attrs[0].value.u32 = SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_SRV6_VPN_SID;
tunnel_map_entry_attrs[1].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP;
tunnel_map_entry_attrs[1].value = srv6_tunnel_encap_map;
tunnel_map_entry_attrs[2].id = SAI_TUNNEL_MAP_ENTRY_ATTR_PREFIX_AGG_ID_KEY;
tunnel_map_entry_attrs[2].value.u32 = prefix_agg_id;
tunnel_map_entry_attrs[3].id = SAI_TUNNEL_MAP_ENTRY_ATTR_SRV6_VPN_SID_VALUE;
tunnel_map_entry_attrs[3].value = vpn_sidlist_id;
saistatus = sai_tunnel_api->create_tunnel_map_entry(&srv6_tunnel_map_entry, switch_id, 4, tunnel_map_entry_attrs);
////// 5. Create (or update existing) a overlay route entry using the nexthop
route_entry.switch_id = 0;
route_entry.vr_id = overlay_vrf;
route_entry.destination.addr_family = SAI_IP_ADDR_FAMILY_IPV4;
route_entry.destination.addr.ip4 = "10.0.0.0"
route_entry.destination.addr.mask = "255.0.0.0";
route_entry_attrs[0].id = SAI_ROUTE_ENTRY_ATTR_NEXT_HOP_ID;
route_entry_attrs[0].value.oid = srv6_nh_grp;
route_entry_attrs[1].id = SAI_ROUTE_ENTRY_ATTR_PREFIX_AGG_ID;
route_entry_attrs[1].value.u32 = prefix_agg_id;
saistatus = sai_route_api->create_route(&route_entry, 2, route_entry_attrs);This section describes the SID Marking feature for SRv6 VPN, which enables VPN SIDs to carry QoS and policy information derived from packet classification. Two new tunnel map types support SID Marking by allowing VPN SID selection based on forwarding class and hierarchical policy lookup.
The forwarding class used for SID Marking is derived from packet classification through one of the following mechanisms:
-
QoS Map Classification: Use
SAI_QOS_MAP_TYPE_DSCP_TO_FORWARDING_CLASSto map incoming packet DSCP values to forwarding class values. The QoS map is attached to the ingress port. -
ACL Classification: Use
SAI_ACL_ENTRY_ATTR_ACTION_SET_FORWARDING_CLASSACL action to set the forwarding class based on ACL match criteria, providing flexible classification beyond DSCP.
When using hierarchical SID Marking with PREFIX_AGG_ID_TO_TUNNEL_MAP_ID, the logical lookup proceeds as follows:
Route Entry (with Prefix Aggregation ID)
│
▼
First Tunnel Map (PREFIX_AGG_ID_TO_TUNNEL_MAP_ID)
│ Key: Prefix Aggregation ID
│ Value: Second Tunnel Map OID
▼
Second Tunnel Map (FORWARDING_CLASS_TO_SRV6_VPN_SID)
│ Key: Forwarding Class (from QoS Map or ACL)
│ Value: VPN SID
▼
Final VPN SID for encapsulation
This two-level indirection enables per-destination, per-QoS-class VPN SID selection, allowing different traffic classes to the same destination to use different VPN SIDs.
Note: While the SAI API models this as a two-level lookup, implementations are likely to collapse the hierarchical lookup into a single lookup using a composite key (e.g., Prefix Aggregation ID + Forwarding Class) for efficiency.
The SAI_TUNNEL_MAP_TYPE_FORWARDING_CLASS_TO_SRV6_VPN_SID tunnel map type implements the core SID Marking capability. It enables VPN SIDs to be marked with QoS information by selecting different VPN SIDs based on forwarding class. The marked VPN SID carries the QoS information end-to-end through the SRv6 network.
typedef enum _sai_tunnel_map_entry_attr_t
{
// ...
// <snip>
// ...
/**
* @brief Forwarding Class key
*
* @type sai_uint32_t
* @flags MANDATORY_ON_CREATE | CREATE_ONLY
* @condition SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE == SAI_TUNNEL_MAP_TYPE_FORWARDING_CLASS_TO_SRV6_VPN_SID
*/
SAI_TUNNEL_MAP_ENTRY_ATTR_FORWARDING_CLASS_KEY = 0x00000013,
/**
* @brief SRv6 VPN SID value
*
* @type sai_ip6_t
* @flags MANDATORY_ON_CREATE | CREATE_ONLY
* @condition SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE == SAI_TUNNEL_MAP_TYPE_FORWARDING_CLASS_TO_SRV6_VPN_SID
*/
SAI_TUNNEL_MAP_ENTRY_ATTR_SRV6_VPN_SID_VALUE = 0x00000011,
// ...
// <snip>
// ...
} sai_tunnel_map_entry_attr_t;The SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_TUNNEL_MAP_ID tunnel map type enables hierarchical, two-level lookup for VPN SID selection. The first level maps a prefix aggregation ID to a second tunnel map, which then maps forwarding class to VPN SID. This allows combining prefix-based routing with QoS-aware SID Marking.
typedef enum _sai_tunnel_map_entry_attr_t
{
// ...
// <snip>
// ...
/**
* @brief Prefix Aggregation ID key
*
* @type sai_uint32_t
* @flags MANDATORY_ON_CREATE | CREATE_ONLY
* @condition SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE == SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_TUNNEL_MAP_ID
*/
SAI_TUNNEL_MAP_ENTRY_ATTR_PREFIX_AGG_ID_KEY = 0x00000010,
/**
* @brief Tunnel Map ID value
*
* @type sai_object_id_t
* @flags MANDATORY_ON_CREATE | CREATE_ONLY
* @objects SAI_OBJECT_TYPE_TUNNEL_MAP
* @condition SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE == SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_TUNNEL_MAP_ID
*/
SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_ID_VALUE = 0x00000012,
// ...
// <snip>
// ...
} sai_tunnel_map_entry_attr_t;The following example demonstrates SID Marking where traffic uses different VPN SIDs based on forwarding class.
// 1. Create QoS Map to derive forwarding class from DSCP (on ingress port)
// This step configures SAI_QOS_MAP_TYPE_DSCP_TO_FORWARDING_CLASS
// (QoS map creation omitted for brevity)
// 2. Create the tunnel map (Forwarding Class -> VPN SID)
tunnel_map_attrs[0].id = SAI_TUNNEL_MAP_ATTR_TYPE;
tunnel_map_attrs[0].value.s32 = SAI_TUNNEL_MAP_TYPE_FORWARDING_CLASS_TO_SRV6_VPN_SID;
saistatus = sai_tunnel_api->create_tunnel_map(&fc_to_sid_map, switch_id, 1, tunnel_map_attrs);
// 2.1 Create tunnel map entries for each forwarding class
// Forwarding Class 0x0 -> VPN SID
tunnel_map_entry_attrs[0].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE;
tunnel_map_entry_attrs[0].value.s32 = SAI_TUNNEL_MAP_TYPE_FORWARDING_CLASS_TO_SRV6_VPN_SID;
tunnel_map_entry_attrs[1].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP;
tunnel_map_entry_attrs[1].value.oid = fc_to_sid_map;
tunnel_map_entry_attrs[2].id = SAI_TUNNEL_MAP_ENTRY_ATTR_FORWARDING_CLASS_KEY;
tunnel_map_entry_attrs[2].value.u32 = 0x0;
tunnel_map_entry_attrs[3].id = SAI_TUNNEL_MAP_ENTRY_ATTR_SRV6_VPN_SID_VALUE;
CONVERT_STR_TO_IPV6(tunnel_map_entry_attrs[3].value.ip6, "fd00:205:2007:fff0:30::");
saistatus = sai_tunnel_api->create_tunnel_map_entry(&fc0_entry, switch_id, 4, tunnel_map_entry_attrs);
// Forwarding Class 0x8 -> VPN SID
tunnel_map_entry_attrs[2].value.u32 = 0x8;
CONVERT_STR_TO_IPV6(tunnel_map_entry_attrs[3].value.ip6, "fd00:205:2007:fff0:38::");
saistatus = sai_tunnel_api->create_tunnel_map_entry(&fc8_entry, switch_id, 4, tunnel_map_entry_attrs);
// Forwarding Class 0xf -> VPN SID
tunnel_map_entry_attrs[2].value.u32 = 0xf;
CONVERT_STR_TO_IPV6(tunnel_map_entry_attrs[3].value.ip6, "fd00:205:2007:fff0:3f::");
saistatus = sai_tunnel_api->create_tunnel_map_entry(&fcf_entry, switch_id, 4, tunnel_map_entry_attrs);
// 3. Create SRv6 tunnel with the FC->SID encap mapper
tunnel_entry_attrs[0].id = SAI_TUNNEL_ATTR_TYPE;
tunnel_entry_attrs[0].value.s32 = SAI_TUNNEL_TYPE_SRV6;
tunnel_entry_attrs[1].id = SAI_TUNNEL_ATTR_ENCAP_SRC_IP;
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[1].value.ip6, "fd00:205:2007::1");
tunnel_entry_attrs[2].id = SAI_TUNNEL_ATTR_UNDERLAY_INTERFACE;
tunnel_entry_attrs[2].value.oid = underlay_rif;
tunnel_entry_attrs[3].id = SAI_TUNNEL_ATTR_ENCAP_MAPPERS;
tunnel_entry_attrs[3].value.objlist.count = 1;
tunnel_entry_attrs[3].value.objlist.list[0] = fc_to_sid_map;
tunnel_entry_attrs[4].id = SAI_TUNNEL_ATTR_PEER_MODE;
tunnel_entry_attrs[4].value.s32 = SAI_TUNNEL_PEER_MODE_P2P;
tunnel_entry_attrs[5].id = SAI_TUNNEL_ATTR_ENCAP_DST_IP;
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[5].value.ip6, "fd00:205:2007::21");
saistatus = sai_tunnel_api->create_tunnel(&srv6_tunnel_id, switch_id, 6, tunnel_entry_attrs);
// 4. Create SRv6 nexthop and route
// (Similar to existing VPN programming, see earlier examples)The following example demonstrates hierarchical SID Marking where a prefix aggregation ID maps to a tunnel map, which then maps forwarding class to VPN SID.
// 1. Create QoS Map to derive forwarding class from DSCP (on ingress port)
// This step configures SAI_QOS_MAP_TYPE_DSCP_TO_FORWARDING_CLASS
// (QoS map creation omitted for brevity)
// 2. Create the inner tunnel map (Forwarding Class -> VPN SID)
tunnel_map_attrs[0].id = SAI_TUNNEL_MAP_ATTR_TYPE;
tunnel_map_attrs[0].value.s32 = SAI_TUNNEL_MAP_TYPE_FORWARDING_CLASS_TO_SRV6_VPN_SID;
saistatus = sai_tunnel_api->create_tunnel_map(&fc_to_sid_map, switch_id, 1, tunnel_map_attrs);
// 2.1 Create tunnel map entries for each forwarding class
// Forwarding Class 0x0 -> VPN SID
tunnel_map_entry_attrs[0].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE;
tunnel_map_entry_attrs[0].value.s32 = SAI_TUNNEL_MAP_TYPE_FORWARDING_CLASS_TO_SRV6_VPN_SID;
tunnel_map_entry_attrs[1].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP;
tunnel_map_entry_attrs[1].value.oid = fc_to_sid_map;
tunnel_map_entry_attrs[2].id = SAI_TUNNEL_MAP_ENTRY_ATTR_FORWARDING_CLASS_KEY;
tunnel_map_entry_attrs[2].value.u32 = 0x0;
tunnel_map_entry_attrs[3].id = SAI_TUNNEL_MAP_ENTRY_ATTR_SRV6_VPN_SID_VALUE;
CONVERT_STR_TO_IPV6(tunnel_map_entry_attrs[3].value.ip6, "fd00:205:2007:fff0:30::");
saistatus = sai_tunnel_api->create_tunnel_map_entry(&fc0_entry, switch_id, 4, tunnel_map_entry_attrs);
// Forwarding Class 0x8 -> VPN SID
tunnel_map_entry_attrs[2].value.u32 = 0x8;
CONVERT_STR_TO_IPV6(tunnel_map_entry_attrs[3].value.ip6, "fd00:205:2007:fff0:38::");
saistatus = sai_tunnel_api->create_tunnel_map_entry(&fc8_entry, switch_id, 4, tunnel_map_entry_attrs);
// Forwarding Class 0xf -> VPN SID
tunnel_map_entry_attrs[2].value.u32 = 0xf;
CONVERT_STR_TO_IPV6(tunnel_map_entry_attrs[3].value.ip6, "fd00:205:2007:fff0:3f::");
saistatus = sai_tunnel_api->create_tunnel_map_entry(&fcf_entry, switch_id, 4, tunnel_map_entry_attrs);
// 3. Create the outer tunnel map (Prefix Aggregation ID -> Tunnel Map)
tunnel_map_attrs[0].id = SAI_TUNNEL_MAP_ATTR_TYPE;
tunnel_map_attrs[0].value.s32 = SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_TUNNEL_MAP_ID;
saistatus = sai_tunnel_api->create_tunnel_map(&prefix_to_map, switch_id, 1, tunnel_map_attrs);
// 3.1 Create entry mapping prefix aggregation ID 10 to the FC->SID map
tunnel_map_entry_attrs[0].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE;
tunnel_map_entry_attrs[0].value.s32 = SAI_TUNNEL_MAP_TYPE_PREFIX_AGG_ID_TO_TUNNEL_MAP_ID;
tunnel_map_entry_attrs[1].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP;
tunnel_map_entry_attrs[1].value.oid = prefix_to_map;
tunnel_map_entry_attrs[2].id = SAI_TUNNEL_MAP_ENTRY_ATTR_PREFIX_AGG_ID_KEY;
tunnel_map_entry_attrs[2].value.u32 = 10; // Prefix Aggregation ID
tunnel_map_entry_attrs[3].id = SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_ID_VALUE;
tunnel_map_entry_attrs[3].value.oid = fc_to_sid_map; // Points to FC->SID map
saistatus = sai_tunnel_api->create_tunnel_map_entry(&prefix_entry, switch_id, 4, tunnel_map_entry_attrs);
// 4. Create SRv6 tunnel with the hierarchical encap mapper
tunnel_entry_attrs[0].id = SAI_TUNNEL_ATTR_TYPE;
tunnel_entry_attrs[0].value.s32 = SAI_TUNNEL_TYPE_SRV6;
tunnel_entry_attrs[1].id = SAI_TUNNEL_ATTR_ENCAP_SRC_IP;
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[1].value.ip6, "fd00:205:2007::1");
tunnel_entry_attrs[2].id = SAI_TUNNEL_ATTR_UNDERLAY_INTERFACE;
tunnel_entry_attrs[2].value.oid = underlay_rif;
tunnel_entry_attrs[3].id = SAI_TUNNEL_ATTR_ENCAP_MAPPERS;
tunnel_entry_attrs[3].value.objlist.count = 1;
tunnel_entry_attrs[3].value.objlist.list[0] = prefix_to_map; // Outer map
tunnel_entry_attrs[4].id = SAI_TUNNEL_ATTR_PEER_MODE;
tunnel_entry_attrs[4].value.s32 = SAI_TUNNEL_PEER_MODE_P2P;
tunnel_entry_attrs[5].id = SAI_TUNNEL_ATTR_ENCAP_DST_IP;
CONVERT_STR_TO_IPV6(tunnel_entry_attrs[5].value.ip6, "fd00:205:2007::21");
saistatus = sai_tunnel_api->create_tunnel(&srv6_tunnel_id, switch_id, 6, tunnel_entry_attrs);
// 5. Create SRv6 nexthop and route with prefix aggregation ID
// Route entry must include SAI_ROUTE_ENTRY_ATTR_PREFIX_AGG_ID = 10
// (Similar to existing VPN programming, see earlier examples)