[warm-reboot] Routing change (add/del) during warm boot automation

[neethajohn]

Description of PR

Testcases for adding and deleting routes during warm-boot. The testcase flow is as follows

• Routes add case

  1. Once the control plane goes down during warm-boot, add 'cnt' number of ipv4 and ipv6 routes on all the neighbors
  2. Verify if the all the routes are received by the DUT
  3. Remove the routes at the end of the test and ensure those routes are removed

• Route delete case

  1. At the start of the test, add 'cnt' number of ipv4 and ipv6 routes on all neighbors. Ensure those routes are received by the DUT
  2. Once the control plane goes down during warm-boot, remove those added routes from all the neighbors
  3. Verify if the routes are removed on the DUT end

Type of change

• [] Bug fix
• [] Testbed and Framework(new/improvement)
• Test case(new/improvement)

Approach

How did you do it?

• warm-reboot-multi-sad-inboot.yml
  - inboot-list contains all the operations that will be done during each warm reboot cycle
  (each item is of type 'inboot_oper:number of routes per address family')
• advanced-reboot.yml
  • populate 'sad_list' with preboot_list or inboot_list or None
• ptf_runner_reboot.yml
  • differentiate between inboot_oper and preboot_oper and pass both of them to the ptf_runner
• advanced-reboot.py
  • preboot and inboot oper are populated as sad_oper. Previous usage of preboot_oper is replaced with sad_oper
  • added support to invoke inboot operation
  • move some code to individual methods
• sad_path.py
  • generating 'cnt' number of routes
  • build a map of bgp peer and curr number of routes. Route cnt is validated using this val + 'cnt' wherever applicable
  • build list of cmds for route addition and deletion on the Arista neighbors
  • support for adding routes prior to route del during warm reboot
  • modifications to setup and verify methods
• arista.py
  - extract the next hop for ipv4 and ipv6 routes
  - method to add/del routes

How did you verify/test it?

Ran the routing change test on t0-64 topology and it passed
Ran regressions on warm-reboot, fast-reboot and preboot list test and they all passed

[yxieca]

@neethajohn this is a great change. I think with the code you have, you could also support adding route during warm reboot and clean up after test?

[neethajohn]

@neethajohn this is a great change. I think with the code you have, you could also support adding route during warm reboot and clean up after test?

@yxieca. That's the way I am doing it currently. if I am passing the 'routing_add:cnt' tag then 'cnt' number of v4 and v6 routes are added during warm reboot and cleaned up at the end of the test (calling the revert() function)

[yxieca]

@neethajohn I thought that was in the code. but PR comments made me think otherwise. So in either case, could this infrastructure support both route add and route delete test?

[neethajohn]

@neethajohn I thought that was in the code. but PR comments made me think otherwise. So in either case, could this infrastructure support both route add and route delete test?

@yxieca, this infrastructure supports both route add and del. I have included both the cases in the test as well

[qiluo-msft]

Commented