Skip to content

Fix in script to avoid orchagent crash when port down followed by fdb…#1340

Merged
daall merged 1 commit intosonic-net:masterfrom
rupesh-k:erspan_script_timing_fix
Jul 2, 2020
Merged

Fix in script to avoid orchagent crash when port down followed by fdb…#1340
daall merged 1 commit intosonic-net:masterfrom
rupesh-k:erspan_script_timing_fix

Conversation

@rupesh-k
Copy link
Contributor

@rupesh-k rupesh-k commented Jul 2, 2020

What I did
Observed that sometimes orchagent is hitting seg fault with below stack when port down followed by FDB delete.

Crash:
Thread 1 "orchagent" received signal SIGSEGV, Segmentation fault.
0x00007f8f742aebb3 in std::_Rb_tree_increment(std::_Rb_tree_node_base const*) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0 0x00007f8f742aebb3 in std::_Rb_tree_increment(std::_Rb_tree_node_base const*) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1 0x000055c07f1f1078 in std::_Rb_tree_const_iterator::operator++ (this=) at /usr/include/c++/6/bits/stl_tree.h:288
#2 FdbOrch::update (this=this@entry=0x55c07f89f6d0, type=, entry=, bridge_port_id=) at fdborch.cpp:235
#3 0x000055c07f1f2002 in FdbOrch::doTask (this=0x55c07f89f6d0, consumer=...) at fdborch.cpp:485
#4 0x000055c07f15d712 in OrchDaemon::start (this=0x55c07f882e80) at orchdaemon.cpp:467
#5 0x000055c07f148322 in main (argc=, argv=) at main.cpp:346

Why I did it
Fixed in script to avoid this as it may be hitting some timing issue.

How I verified it
Ran script 50 times without any issue.

Details if related

… delete

Signed-off-by: Rupesh Kumar <rupesh-k.kumar@broadcom.com>
@rupesh-k
Copy link
Contributor Author

rupesh-k commented Jul 2, 2020

Hi @daall

Observed in swss-build-pr the tests failed, on further investigation found that it is hitting the crash in fdborch when port down is followed by fdb delete.

Fixing in script to avoid this.

I am not sure whom to follow up on the fdb orch crash.

Thanks

@daall
Copy link
Contributor

daall commented Jul 2, 2020

retest this please

@daall
Copy link
Contributor

daall commented Jul 2, 2020

Thanks for the quick find and fix @rupesh-k! Can you open an issue in the sonic-buildimage repo with the vs tag?

@lguohan FYI

@daall daall merged commit 9b13120 into sonic-net:master Jul 2, 2020
@rupesh-k rupesh-k deleted the erspan_script_timing_fix branch July 3, 2020 05:37
EdenGri pushed a commit to EdenGri/sonic-swss that referenced this pull request Feb 28, 2022
Janetxxx pushed a commit to Janetxxx/sonic-swss that referenced this pull request Nov 10, 2025
… delete (sonic-net#1340)

Signed-off-by: Rupesh Kumar <rupesh-k.kumar@broadcom.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants