Description
[Sflow] : Kernel panic is seen upon sending Icmp packets with different packet size to interface in which sflow is enabled
admin@vlab-01:~$ show sflowsFlow Global Information:
sFlow Admin State: up
sFlow Polling Interval: 60
sFlow AgentID: default 1 Collectors configured:
Name: collector_1 IP addr: 192.168.0.3UDP port: 6343
admin@vlab-01:~$ show sflow interfacesFlow interface configurations
+-------------+---------------+-----------------+
| Interface | Admin State | Sampling Rate |
+=============+===============+=================+
| Ethernet112 | up | 512 |
ping the DUT interface ip from peer continuously with different packet size. Check for kernel crash
admin@vlab-01:~$ timed out waiting for input: auto-logoutDebian GNU/Linux 9 vlab-01 ttyS0vlab-01 login: [16645.523275] skbuff: skb_over_panic: text:ffffffffc0577382 len:196 put:136 head:ffff90aae8d9a400 data:ffff90aae8d9a400 tail:0xc4 end:0xc0 dev:<NULL>
[16645.539991] ------------[ cut here ]------------
[16645.546260] kernel BUG at /sonic/src/sonic-linux-kernel/linux-4.9.168/net/core/skbuff.c:105!
[16645.552584] invalid opcode: 0000 [#1] SMP
[16645.554759] Modules linked in: cls_matchall sch_ingress tun 8021q garp mrp dummy team_mode_loadbalance team ip6table_mangle xt_TCPMSS xt_tcpudp iptable_mangle xt_conntrack nf_nat nf_conntrack_netlink nfnetlink nf_conntrack xfrm_user xfrm_algo joydev kvm_intel kvm irqbypass pcspkr evdev serio_raw bonding virtio_balloon shpchp button ebtable_broute bridge stp llc ebtable_nat ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter act_sample psample ip_tables x_tables autofs4 loop ext4 crc16 jbd2 crc32c_generic fscrypto ecb glue_helper lrw gf128mul ablk_helper cryptd aes_x86_64 mbcache nls_utf8 nls_cp437 nls_ascii vfat fat overlay squashfs virtio_blk psmouse ata_generic ata_piix virtio_pci uhci_hcd virtio_ring virtio ehci_hcd libata floppy e1000 i2c_piix4 scsi_mod usbcore usb_common
[16645.599686] CPU: 1 PID: 16 Comm: ksoftirqd/1 Not tainted 4.9.0-9-2-amd64 #1 Debian 4.9.168-1+deb9u5
[16645.604164] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[16645.607062] task: ffff90ab38240240 task.stack: ffffaa430039c000
[16645.609611] RIP: 0010:[<ffffffffbb6faca0>] [<ffffffffbb6faca0>] skb_panic+0x60/0x70
[16645.614309] RSP: 0018:ffffaa430039fbd8 EFLAGS: 00010286
[16645.617380] RAX: 0000000000000087 RBX: 0000000000000082 RCX: 0000000000000000
[16645.621205] RDX: 0000000000000000 RSI: ffff90ab3cd10648 RDI: ffff90ab3cd10648
[16645.624596] RBP: ffff90aae5101c00 R08: 0000000000000000 R09: 00000000bc0ba601
[16645.628025] R10: 0000000000000014 R11: ffffffffbc0ba5ad R12: ffffaa430039fc1c
[16645.631757] R13: ffff90aae790b400 R14: 0000000000000000 R15: ffff90aae74e6940
[16645.635100] FS: 0000000000000000(0000) GS:ffff90ab3cd00000(0000) knlGS:0000000000000000
[16645.638971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16645.642169] CR2: 00007fb73002d118 CR3: 000000003b0d8000 CR4: 0000000000000670
[16645.646171] Stack:
[16645.647368] ffff90aae8d9a400 00000000000000c4 00000000000000c0 ffffffffbbc15029
[16645.651822] ffffffffbb6fb279 ffffffffc0577382 00000200e5101c00 ffff90aae8d9a414
[16645.656294] 00000095bbedc040 4e4a7bd24042a724 ffff90aae53b6a00 ffff90aae790b400
[16645.660722] Call Trace:
[16645.662107] [<ffffffffbb6fb279>] ? skb_put+0x49/0x50
[16645.665032] [<ffffffffc0577382>] ? psample_sample_packet+0x182/0x2d0 [psample]
[16645.669089] [<ffffffffc059657f>] ? tcf_sample_act+0xbf/0x1a0 [act_sample]
[16645.672564] [<ffffffffbb74255c>] ? tcf_action_exec+0x4c/0x80
[16645.674975] [<ffffffffbb73de52>] ? tc_classify+0x72/0x100
[16645.678160] [<ffffffffbb710f51>] ? __netif_receive_skb_core+0x461/0xa40
[16645.682964] [<ffffffffbb81b7b0>] ? __switch_to_asm+0x40/0x70
[16645.686507] [<ffffffffbb81b7a4>] ? __switch_to_asm+0x34/0x70
[16645.690525] [<ffffffffbb81b7b0>] ? __switch_to_asm+0x40/0x70
[16645.694761] [<ffffffffbb81b7a4>] ? __switch_to_asm+0x34/0x70
[16645.698873] [<ffffffffbb712668>] ? process_backlog+0x88/0x130
[16645.702652] [<ffffffffbb711df6>] ? net_rx_action+0x246/0x380
[16645.705900] [<ffffffffbb81f36d>] ? __do_softirq+0x10d/0x2b0
[16645.709678] [<ffffffffbb29d560>] ? sort_range+0x20/0x20
[16645.712121] [<ffffffffbb27ff5e>] ? run_ksoftirqd+0x1e/0x40
[16645.715454] [<ffffffffbb29d66e>] ? smpboot_thread_fn+0x10e/0x160
[16645.719348] [<ffffffffbb299dd9>] ? kthread+0xd9/0xf0
[16645.722980] [<ffffffffbb299d00>] ? kthread_park+0x60/0x60
[16645.727181] [<ffffffffbb81b837>] ? ret_from_fork+0x57/0x70
[16645.730174] Code: cc 00 00 00 48 89 44 24 10 8b 87 c8 00 00 00 48 89 44 24 08 48 8b 87 d8 00 00 00 48 c7 c7 88 3c c6 bb 48 89 04 24 e8 6c 5f c8 ff <0f> 0b 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 e9 7b 70 e4 ff
[16645.748125] RIP [<ffffffffbb6faca0>] skb_panic+0x60/0x70
[16645.751091] RSP <ffffaa430039fbd8>
[16645.754225] ---[ end trace e5348bf22a10a918 ]---
[16645.756500] Kernel panic - not syncing: Fatal exception in interrupt
[16645.760491] Kernel Offset: 0x3a200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[16645.765845] Rebooting in 10 seconds..
Steps to reproduce the issue:
- configure ip address on an interface
- enable sflow on that interface
- ping the interface IP address with different packet size from peer.
- Check for crash
Describe the results you received:
Kenel panic is seen
Describe the results you expected:
Sflow should the sample packets and send to collector
Additional information you deem important (e.g. issue happens only occasionally):
TESTED in SONIC-VS
**Output of `show version`:**
root@vlab-01:~# show vers
SONiC Software Version: SONiC.HEAD.188-4458efbd
Distribution: Debian 9.11
Kernel: 4.9.0-9-2-amd64
Build commit: 4458efbd
Build date: Wed Dec 18 08:42:30 UTC 2019
Built by: johnar@jenkins-worker-11
Platform: x86_64-kvm_x86_64-r0
HwSKU: Force10-S6000
ASIC: vs
Serial Number: 000000
Uptime: 10:03:26 up 58 min, 1 user, load average: 0.41, 0.27, 0.30
Docker images:
REPOSITORY TAG IMAGE ID SIZE
docker-syncd-vs HEAD.188-4458efbd 495cc11160dd 291MB
docker-syncd-vs latest 495cc11160dd 291MB
docker-platform-monitor HEAD.188-4458efbd 07626700bce9 333MB
docker-platform-monitor latest 07626700bce9 333MB
docker-fpm-frr HEAD.188-4458efbd 5b99dc1e2a92 325MB
docker-fpm-frr latest 5b99dc1e2a92 325MB
docker-lldp-sv2 HEAD.188-4458efbd f40b3e9c0a24 303MB
docker-lldp-sv2 latest f40b3e9c0a24 303MB
docker-dhcp-relay HEAD.188-4458efbd f56f85f26935 290MB
docker-dhcp-relay latest f56f85f26935 290MB
docker-database HEAD.188-4458efbd 452bb9eae044 282MB
docker-database latest 452bb9eae044 282MB
docker-sflow HEAD.188-4458efbd a7b040a3c4f1 305MB
docker-sflow latest a7b040a3c4f1 305MB
docker-teamd HEAD.188-4458efbd 05a8b6083599 305MB
docker-teamd latest 05a8b6083599 305MB
docker-snmp-sv2 HEAD.188-4458efbd 35d2e24c3598 339MB
docker-snmp-sv2 latest 35d2e24c3598 339MB
docker-orchagent HEAD.188-4458efbd 71397b40a481 323MB
docker-orchagent latest 71397b40a481 323MB
docker-sonic-telemetry HEAD.188-4458efbd 8cca2b75a13d 305MB
docker-sonic-telemetry latest 8cca2b75a13d 305MB
docker-router-advertiser HEAD.188-4458efbd 820d05e70b3c 282MB
docker-router-advertiser latest 820d05e70b3c 282MB
root@vlab-01:~#
**Attach debug file `sudo generate_dump`:**
```
(paste your output here)
```
Description
[Sflow] : Kernel panic is seen upon sending Icmp packets with different packet size to interface in which sflow is enabled
ping the DUT interface ip from peer continuously with different packet size. Check for kernel crash
Steps to reproduce the issue:
Describe the results you received:
Kenel panic is seen
Describe the results you expected:
Sflow should the sample packets and send to collector
Additional information you deem important (e.g. issue happens only occasionally):
TESTED in SONIC-VS