Updating arp responder.py to capture and use vlan from packet

[dgsudharsan]

Fixes based on UT

Description of PR

Summary:
Updated arp responder.py to use libpcap socket as low level l2caplisten socket so that arp_responder gets vlan tag from the packet.
The vlan tag is then used in the arp response.
With this change an interface can have multiple vlans. Earlier there was one vlan per interface which is picked from /tmp/from_t1.json

PLEASE DO NOT MERGE THIS UNTIL sonic-net/sonic-buildimage#3731 is merged

Fixes # (issue)

Type of change

• [] Bug fix
• [] Testbed and Framework(new/improvement)
• [] Test case(new/improvement)

Approach

How did you do it?

Updated docker-ptf to include python-libpcap.
After setting conf.use_pcap = True
print the value of conf.L2listen
and the value is
<L2pcapListenSocket: read packets at layer 2 using libpcap>

Without the configuration the value printed would be
<L2ListenSocket: read packets at layer 2 using Linux PF_PACKET sockets>

Use scapy send and recv instead of traditional socket send and recv API calls.

How did you verify/test it?

After this change verify by running existing test cases. All the test cases that uses arp_responder.py passed.
Additional ran the script separately to ensure vlan tag is received in a packet during arp request.
Below is the sample received packet
ffffffffffff4c7625f446808100000a080600010800060400014c7625f446800101010100000000000001010103

The packet was sent on vlan 10 and "8100 000a" confirms vlan 10 is present in packet received.

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[prsunny]

@dgsudharsan , do we need to make any change to existing PTF docker for this to run?

[dgsudharsan]

@dgsudharsan , do we need to make any change to existing PTF docker for this to run?

Yes. It has been done and changes have been merged
sonic-net/sonic-buildimage#3731

[prsunny]

@volodymyrsamotiy , can you check this?

[volodymyrsamotiy]

Looks like still not fully working when there are multiple VLAN interfaces/neighbors on one physical port. (when run VNET test with default parameters all works good)
For example, I am trying to run VNET test with the parameters as shown below and as a result it fails due to not neighbors resolved.

ansible-playbook test_sonic.yml -i inventory --limit <testbed> -e testbed_name=<testbed_name> -e testbed_type=t0 -e testcase_name=vnet_vxlan -e cleanup=False -e "num_vnet=32 num_routes=2000 num_endpoints=1000"

Below is the neighbors dump snippet from DUT during running above test.

root@sonic# ip -4 neigh | grep Vlan
30.1.10.102 dev Vlan3001 lladdr 24:8a:07:3e:0c:02 REACHABLE
30.27.10.101 dev Vlan3027  FAILED
30.18.10.101 dev Vlan3018 lladdr 24:8a:07:3e:0c:13 REACHABLE
30.15.10.101 dev Vlan3015 lladdr 24:8a:07:3e:0c:10 REACHABLE
30.28.10.101 dev Vlan3028 lladdr 24:8a:07:3e:0c:06 REACHABLE
30.16.10.101 dev Vlan3016 lladdr 24:8a:07:3e:0c:11 REACHABLE
30.14.10.101 dev Vlan3014 lladdr 24:8a:07:3e:0c:0f REACHABLE
30.10.10.101 dev Vlan3010 lladdr 24:8a:07:3e:0c:0b REACHABLE
30.18.10.102 dev Vlan3018 lladdr 24:8a:07:3e:0c:13 REACHABLE
30.17.10.102 dev Vlan3017 lladdr 24:8a:07:3e:0c:12 REACHABLE
30.21.10.102 dev Vlan3021 lladdr 24:8a:07:3e:0c:16 REACHABLE
30.30.10.101 dev Vlan3030  FAILED
30.8.10.101 dev Vlan3008  FAILED
30.26.10.101 dev Vlan3026  FAILED
30.14.10.102 dev Vlan3014 lladdr 24:8a:07:3e:0c:0f REACHABLE
30.24.10.101 dev Vlan3024  FAILED
30.13.10.102 dev Vlan3013 lladdr 24:8a:07:3e:0c:0e REACHABLE
30.10.10.102 dev Vlan3010 lladdr 24:8a:07:3e:0c:0b REACHABLE
30.5.10.101 dev Vlan3005  FAILED
30.21.10.101 dev Vlan3021 lladdr 24:8a:07:3e:0c:16 REACHABLE
30.31.10.101 dev Vlan3031 lladdr 24:8a:07:3e:0c:09 REACHABLE
30.9.10.102 dev Vlan3009 lladdr 24:8a:07:3e:0c:0a REACHABLE
30.26.10.102 dev Vlan3026  FAILED
...

[dgsudharsan]

Hi,
I am not sure if these failures are consequences of the arp_responder changes. If the changes do not work I assume they don't work fully. But what we see here is some gets resolved while some don't. Is it due to some timing issue? Are there any logs are debugs? Also is arp responder running in this scenario during tests or gets exited in some scenario?

[volodymyrsamotiy]

I tried it manually and observed the same behavior. ARP responder was running all the time and I applied JSON config with the neighbors, Some neighbors were not resolved, it always the same number and the same neighbors. Looks like it happens for the neighbors which are configured on different VLAN interfaces but if interfaces are configured on the same physical port.

For example, looking at VNET config, we have the following configured:

{
    "VLAN_MEMBER": {
        "Vlan3006|Ethernet28": {
            "tagging_mode": "tagged"
        },
        "Vlan3029|Ethernet28": {
            "tagging_mode": "tagged"
        },
...
    "NEIGH": {
        "Vlan3006|30.6.10.101": {
            "family": "IPv4"
        },
        "Vlan3006|30.6.10.102": {
            "family": "IPv4"
        },
        "Vlan3029|30.29.10.101": {
            "family": "IPv4"
        },
        "Vlan3029|30.29.10.102": {
            "family": "IPv4"
        },
...
}

Below is the part of ARP responder config from the PTF container:

{..."eth7@3006": ["30.6.10.101", "30.6.10.102"], "eth7@3029": ["30.29.10.101", "30.29.10.102"]...}

And with such config we have always only part of neighbors resolved (always the same):

admin@sonic# ip -4 neigh
30.6.10.102 dev Vlan3006  FAILED
30.6.10.101 dev Vlan3006  FAILED
30.29.10.101 dev Vlan3029 lladdr 98:03:9b:82:ba:07 REACHABLE
30.29.10.102 dev Vlan3029 lladdr 98:03:9b:82:ba:07 REACHABLE

Just a guess, It could be that in ARP responder script, when it reads config file, first neighbors are stored for some port and if then we got some new neighbors for this port they are just overrided with the latest ones and as a result we lose previous entries.

[dgsudharsan]

Hi,
I think there is a little bit of configuration change needed. We need not have vlan's any more
so the json generated should have been
{..."eth7": ["30.6.10.101", "30.6.10.102","30.29.10.101", "30.29.10.102"],..}

[volodymyrsamotiy]

Now it works, thanks