[CRASH] crash when unblocking a client on timeout #2111
Description
Crash report
#0 0x00007ffff61fe2a9 in syscall () from /lib64/libc.so.6
(gdb) bt
#0 0x00007ffff61fe2a9 in syscall () from /lib64/libc.so.6
#1 0x00000000004523eb in ThreadsManager_runOnThreads (tids=0x7fffffffc0d0, tids_len=4, callback=0x5d7d48 <collect_stacktrace_data>) at threads_mngr.c:98
#2 0x00000000005d8130 in writeStacktraces (fd=1, uplevel=3) at debug.c:1729
#3 0x00000000005d8729 in logStackTrace (eip=0x0, uplevel=3, current_thread=0) at debug.c:1816
#4 0x00000000005d72b6 in _serverPanic (file=0x7f0080 "blocked.c", line=279, msg=0x7f0360 "Unknown btype in replyToBlockedClientTimedOut().") at debug.c:1158
#5 0x00000000006636d8 in replyToBlockedClientTimedOut (c=0x61600000e480) at blocked.c:279
#6 0x00000000006661e7 in unblockClientOnTimeout (c=0x61600000e480) at blocked.c:738
#7 0x00000000004e7856 in clientUnblockCommand (c=0x61600001da80) at networking.c:4334
#8 0x000000000048f1aa in call (c=0x61600001da80, flags=3) at server.c:3733
#9 0x00000000004931e6 in processCommand (c=0x61600001da80) at server.c:4359
#10 0x00000000004de07e in processCommandAndResetClient (c=0x61600001da80) at networking.c:3142
#11 0x00000000004de7fa in processInputBuffer (c=0x61600001da80) at networking.c:3270
#12 0x00000000004df248 in readQueryFromClient (conn=0x60600000a400) at networking.c:3375
#13 0x0000000000729304 in callHandler (conn=0x60600000a400, handler=0x4df15a <readQueryFromClient>) at connhelpers.h:79
#14 0x000000000072aaf8 in connSocketEventHandler (el=0x60c00000bf80, fd=89, clientData=0x60600000a400, mask=1) at socket.c:301
#15 0x0000000000462d14 in aeProcessEvents (eventLoop=0x60c00000bf80, flags=27) at ae.c:486
#16 0x000000000046349e in aeMain (eventLoop=0x60c00000bf80) at ae.c:543
#17 0x00000000004a487c in main (argc=1, argv=0x7fffffffe068) at server.c:7212
= VALKEY BUG REPORT START: Cut & paste starting from here ===
6160:M 16 May 2025 14:22:10.387 # ------------------------------------------------
6160:M 16 May 2025 14:22:10.387 # !!! Software Failure. Press left mouse button to continue
6160:M 16 May 2025 14:22:10.387 # Guru Meditation: Unknown btype in replyToBlockedClientTimedOut(). #blocked.c:279
------ STACK TRACE ------
Thread 2 "bio_close_file" received signal SIGUSR2, User defined signal 2.
[Switching to Thread 0x7fffec530700 (LWP 6167)]
0x00007ffff64ce377 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00007ffff64ce377 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00000000006290e9 in bioProcessBackgroundJobs (arg=0x0) at bio.c:233
#2 0x00007ffff64c844b in start_thread () from /lib64/libpthread.so.0
#3 0x00007ffff620352f in clone () from /lib64/libc.so.6
(gdb) thr 1
[Switching to thread 1 (Thread 0x7ffff7fe0c00 (LWP 6160))]
#0 0x00007ffff61fe2a9 in syscall () from /lib64/libc.so.6
...
.=================================================================
==6160==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000000bc23d8 at pc 0x7ffff6ea121a bp 0x7fffffefc3b0 sp 0x7fffffefbb58
READ of size 221184 at 0x000000bc23d8 thread T0
#0 0x7ffff6ea1219 (/lib64/libasan.so.4+0x76219)
#1 0x62edba in memtest_preserving_test src/memtest.c:304
#2 0x5d9829 in memtest_test_linux_anonymous_maps src/debug.c:1983
#3 0x5d9ae2 in doFastMemoryTest src/debug.c:2026
#4 0x5dafae in printCrashReport src/debug.c:2230
#5 0x5d72c3 in _serverPanic src/debug.c:1162
#6 0x6636d7 in replyToBlockedClientTimedOut src/blocked.c:279
#7 0x6661e6 in unblockClientOnTimeout src/blocked.c:738
#8 0x4e7855 in clientUnblockCommand src/networking.c:4334
#9 0x48f1a9 in call src/server.c:3733
#10 0x4931e5 in processCommand src/server.c:4359
#11 0x4de07d in processCommandAndResetClient src/networking.c:3142
#12 0x4de7f9 in processInputBuffer src/networking.c:3270
#13 0x4df247 in readQueryFromClient src/networking.c:3375
#14 0x729303 in callHandler src/connhelpers.h:79
#15 0x72aaf7 in connSocketEventHandler src/socket.c:301
#16 0x462d13 in aeProcessEvents src/ae.c:486
#17 0x46349d in aeMain src/ae.c:543
#18 0x4a487b in main src/server.c:7212
#19 0x7ffff6135139 in __libc_start_main (/lib64/libc.so.6+0x21139)
#20 0x452029 in _start (/localsrc/valkey-server+0x452029)
0x000000bc23d8 is located 0 bytes to the right of global variable 'used_memory_thread_padded' defined in 'zmalloc.c:103:57' (0xbc1b80) of size 2136
0x000000bc23d8 is located 40 bytes to the left of global variable 'total_active_threads' defined in 'zmalloc.c:109:19' (0xbc2400) of size 4
SUMMARY: AddressSanitizer: global-buffer-overflow (/lib64/libasan.so.4+0x76219)
Shadow bytes around the buggy address:
0x000080170420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080170430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080170440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080170450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x000080170460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x000080170470: 00 00 00 00 00 00 00 00 00 00 00[f9]f9 f9 f9 f9
0x000080170480: 04 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
0x000080170490: 04 f9 f9 f9 f9 f9 f9 f9 00 00 00 00 f9 f9 f9 f9
0x0000801704a0: 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 00 00
0x0000801704b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0000801704c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cbAdditional information
Credits to @uriyage for locating this with his fuzzer testing
However we did not yet dive deep or found a reproduction
Metadata
Metadata
Assignees
Labels
No labels