Enhance vnet_vxlan for scaling of vnets and routes

[manjuv1]

Description of PR

Summary:
Enhance vnet_vxlan for scaling of Vnets/Routes/Endpoints

Type of change

• [] Bug fix
• [] Testbed and Framework(new/improvement)
• [Y] Test case(new/improvement)

Approach

How did you do it?

Enhance vnet_vxlan test to scale to a configured number of vnets/routes/endpoints. If none of these configurations are passed, the default 8 Vnets will be created.

How did you verify/test it?

ansible-playbook (generated vnet_config.yml)
Pass the new parameters to ansible-playbook for the scale test: -e "num_vnet=51 num_routes=550 num_endpoints=200"
For the existing test with 8 Vnets, no need to pass the above parameters.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[msftclas]

All CLA requirements met.

[volodymyrsamotiy]

These changes break "Serv2Serv" test case, it fails because no local routes installed on DUT, as a result traffic is not going between peering VNETs.
For example test generates following config for Vnet3 and Vnet4 peering VNETs (vnet.route.json):

[
...
    {
       "VNET_ROUTE_TABLE:Vnet3:4.4.10.0/24": {
           "ifname": "Vlan3004"
       },
       "OP": "SET"
    },
    {
       "VNET_ROUTE_TABLE:Vnet4:4.4.10.0/24": {
           "ifname": "Vlan3004"
       },
       "OP": "SET"
    },
...
]

But for Vnet3 and Vnet4 peers test sends traffic to 30.4.10.101 IP address and it fails since there are no such routes (instead there are listed above routes).
If manually install correct routes, which are shown below traffic goes as expected.

[
...
    {
       "VNET_ROUTE_TABLE:Vnet3:30.4.10.0/24": {
           "ifname": "Vlan3004"
       },
       "OP": "SET"
    },
    {
       "VNET_ROUTE_TABLE:Vnet4:30.4.10.0/24": {
           "ifname": "Vlan3004"
       },
       "OP": "SET"
    },
...
]

[prsunny]

Can we have this set to max 128?

[manjuv1]

Can you please review if the changes to limit to 128 are fine? I still have to validate and see if the total routes exceeds the configured "num_routes" with the introduction of new sets of peers.

[prsunny]

Changes looks ok to me.

[volodymyrsamotiy]

After latest changes Serv2Serv works good now, but found another issue that in some cases not all required neighbors are configured correctly and as a result test fails.
Issue is observed when try to check 128 VNETs, when run playbook with the num_vnet=128, for example:

$ ansible-playbook test_sonic.yml -i inventory -e testbed_name=<testbed_name> -e testbed_type=t0 -e testcase_name=vnet_vxlan -e "num_vnet=128 num_routes=1024 num_endpoints=512"

After test failed go to DUT and check the following:

root@sonic:/home/admin# cat /tmp/vnet.nbr.json | grep Vlan | wc -l
192
root@sonic:/home/admin# redis-cli -n 4 keys "NEIGH*Vlan*" | wc -l
192
root@sonic:/home/admin# redis-cli -n 0 keys "NEIGH*Vlan*" | wc -l
37

As shown above test generates config vnet.nbr.json with the 192 neighbors to be configured and we do observe 192 entries in CONFIG_DB but to APPLICATION_DB only 37 neighbors was configured.
As a result some test cases are failing since many neighbors are missed and there is no way for traffic to pass.

(BTW, with default values test passes and also passes if less VNETs are specified, e.g. -e "num_vnet=16 num_routes=1024 num_endpoints=512", since less neighbors are configure in such cases and all applied correctly)

[volodymyrsamotiy]

Issue with number of neighbor entries is not related to the test and should be handled separately, so approving the PR.