KeyDB 5.1.1 periodically crashes with assertion failed

[sisoftrg]

I migrated from Redis few days ago, code from RELEASE_5. All work well except such crashes. My setup is active-active replication between 2 nodes, many write (non-overlaping) and read operations on both nodes.

Full log of last crash:
bug.txt

From dmesg:

[29821622.742679] keydb-server[3880]: segfault at ffffffffffffffff ip 000000000048dace sp 00007f78c4bfc0d0 error 7 in keydb-server[400000+1d0000]
[29854550.593999] keydb-server[27184]: segfault at 7f51602f ip 00007f4f602fbe08 sp 00007f4f602fbe10 error 6
[30118136.459673] traps: keydb-server[23276] general protection ip:4c0857 sp:7f22c91fc5a8 error:0 in keydb-server[400000+1d0000]
[30125101.951851] keydb-server[29442]: segfault at ffffffffffffffff ip 000000000048d488 sp 00007f43cf9fc4d0 error 7 in keydb-server[400000+1d0000]
[30132379.101028] keydb-server[31173]: segfault at ffffffffffffffff ip 000000000048d488 sp 00007f707f1fc4d0 error 7 in keydb-server[400000+1d0000]
[30218350.872854] keydb-server[19147]: segfault at ffffffffffffffff ip 000000000048d488 sp 00007f23fb63f4d0 error 7 in keydb-server[400000+1d0000]

Address of assertion:

$ addr2line -e /usr/local/bin/keydb-server 450963
/usr/src/KeyDB/src/networking.cpp:2152

    serverAssert(c->iel == ielFromEventLoop(el));

Please let me know if you need something more.

[JohnSully]

EDIT: Missed your link to the log

[JohnSully]

I mixed this up with a different bug. This bug is still active.

[JohnSully]

We're crashing because the client pointer is NULL. All the codepaths that could lead here should be crashing much earlier so I'm still working on sorting out how this could happen.

Are you running any modules?

[JohnSully]

Great news, I was able to repro this and get a fix. The fix is in unstable and will be included in the next release. unstable docker rebuilds nightly if you would like to try it.

The issue is caused if a new client connects around the time we finish an AOF rewrite. A dangling fd for the AOF rewrite gets used for the new client resulting in the function being called with a NULL client pointer.