-
Notifications
You must be signed in to change notification settings - Fork 957
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Crash report
=== VALKEY BUG REPORT START: Cut & paste starting from here ===
1:M 01 Oct 2025 03:18:36.984 # valkey 8.1.3 crashed by signal: 11, si_code: 1
1:M 01 Oct 2025 03:18:36.984 # Accessing address: 0x4
1:M 01 Oct 2025 03:18:36.984 # Crashed running the instruction at: 0x585ed780fb7d
------ STACK TRACE ------
EIP:
valkey-server *:6379(expireScanCallback+0xd)[0x585ed780fb7d]
1 valkey-server *
/lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7b02ce2c1050]
valkey-server *:6379(expireScanCallback+0xd)[0x585ed780fb7d]
valkey-server *:6379(activeExpireCycle+0xb84)[0x585ed7810824]
valkey-server *:6379(databasesCron+0x4c2)[0x585ed76e6632]
valkey-server *:6379(serverCron+0x74b)[0x585ed76e721b]
valkey-server *:6379(+0x8b0c4)[0x585ed76d00c4]
valkey-server *:6379(aeMain+0x130)[0x585ed76da1b0]
valkey-server *:6379(main+0x57a)[0x585ed76ce6da]
/lib/x86_64-linux-gnu/libc.so.6(+0x2724a)[0x7b02ce2ac24a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7b02ce2ac305]
valkey-server *:6379(_start+0x21)[0x585ed76cffa1]
37 bio_aof
/lib/x86_64-linux-gnu/libc.so.6(+0x85f16)[0x7b02ce30af16]
/lib/x86_64-linux-gnu/libc.so.6(pthread_cond_wait+0x1e8)[0x7b02ce30d5d8]
valkey-server *:6379(bioProcessBackgroundJobs+0x197)[0x585ed77b9087]
/lib/x86_64-linux-gnu/libc.so.6(+0x891f5)[0x7b02ce30e1f5]
/lib/x86_64-linux-gnu/libc.so.6(__clone+0x40)[0x7b02ce38db00]
36 bio_close_file
/lib/x86_64-linux-gnu/libc.so.6(+0x85f16)[0x7b02ce30af16]
/lib/x86_64-linux-gnu/libc.so.6(pthread_cond_wait+0x1e8)[0x7b02ce30d5d8]
valkey-server *:6379(bioProcessBackgroundJobs+0x197)[0x585ed77b9087]
/lib/x86_64-linux-gnu/libc.so.6(+0x891f5)[0x7b02ce30e1f5]
/lib/x86_64-linux-gnu/libc.so.6(__clone+0x40)[0x7b02ce38db00]
38 bio_lazy_free
/lib/x86_64-linux-gnu/libc.so.6(+0x85f16)[0x7b02ce30af16]
/lib/x86_64-linux-gnu/libc.so.6(pthread_cond_wait+0x1e8)[0x7b02ce30d5d8]
valkey-server *:6379(bioProcessBackgroundJobs+0x197)[0x585ed77b9087]
/lib/x86_64-linux-gnu/libc.so.6(+0x891f5)[0x7b02ce30e1f5]
/lib/x86_64-linux-gnu/libc.so.6(__clone+0x40)[0x7b02ce38db00]
4/4 expected stacktraces.
------ STACK TRACE DONE ------
------ REGISTERS ------
1:M 01 Oct 2025 03:18:36.992 #
RAX:0000000000000005 RBX:00007b02bce2a740
RCX:0000000000000002 RDX:0000000000000004
RDI:00007ffdb51b2960 RSI:0000000000000000
RBP:00007ffdb51b2860 RSP:00007ffdb51b2840
R8 :fffffffffffff550 R9 :00007b02cde00c70
R10:00007b02ce280768 R11:00000bf15e20ad68
R12:0000000000000002 R13:0000000000000001
R14:00007b02cde47480 R15:0000000000000001
RIP:0000585ed780fb7d EFL:0000000000010202
CSGSFS:002b000000000033
1:M 01 Oct 2025 03:18:36.992 * hide-user-data-from-log is on, skip logging stack content to avoid spilling user data.
------ INFO OUTPUT ------
# Server
redis_version:7.2.4
server_name:valkey
valkey_version:8.1.3
valkey_release_stage:ga
redis_git_sha1:00000000
redis_git_dirty:1
redis_build_id:7a72b458650d7456
server_mode:standalone
os:Linux 6.8.0-1030-aws x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
gcc_version:12.2.0
process_id:1
process_supervised:no
run_id:1a2bcb7aaa706ab39fff399e069f0fd307891534
tcp_port:6379
server_time_usec:1759288716984463
uptime_in_seconds:942904
uptime_in_days:10
hz:10
configured_hz:10
clients_hz:10
lru_clock:14458252
executable:/valkey-server
config_file:
io_threads_active:0
availability_zone:
listener0:name=tcp,bind=*,bind=-::*,port=6379
# Clients
connected_clients:66
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:20480
client_recent_max_output_buffer:20504
blocked_clients:0
tracking_clients:0
pubsub_clients:3
watching_clients:0
clients_in_timeout_table:0
total_watched_keys:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
paused_reason:none
paused_actions:none
paused_timeout_milliseconds:0
# Memory
used_memory:1550396168
used_memory_human:1.44G
used_memory_rss:1599864832
used_memory_rss_human:1.49G
used_memory_peak:1585515560
used_memory_peak_human:1.48G
used_memory_peak_perc:97.78%
used_memory_overhead:61912320
used_memory_startup:889376
used_memory_dataset:1488483848
used_memory_dataset_perc:96.06%
allocator_allocated:1550770928
allocator_active:1567567872
allocator_resident:1595318272
allocator_muzzy:0
total_system_memory:8158130176
total_system_memory_human:7.60G
used_memory_lua:32768
used_memory_vm_eval:32768
used_memory_lua_human:32.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:33792
used_memory_vm_total:66560
used_memory_vm_total_human:65.00K
used_memory_functions:224
used_memory_scripts:224
used_memory_scripts_human:224B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.01
allocator_frag_bytes:16796944
allocator_rss_ratio:1.02
allocator_rss_bytes:27750400
rss_overhead_ratio:1.00
rss_overhead_bytes:4546560
mem_fragmentation_ratio:1.03
mem_fragmentation_bytes:49470680
mem_not_counted_for_evict:7880
mem_replication_backlog:10485864
mem_total_replication_buffers:10532872
mem_clients_slaves:47112
mem_clients_normal:254504
mem_cluster_links:0
mem_aof_buffer:1792
mem_allocator:jemalloc-5.3.0
mem_overhead_db_hashtable_rehashing:0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:28
# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:36199211
rdb_bgsave_in_progress:0
rdb_last_save_time:1758345812
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:7
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:31232000
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:1065884
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:7
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:101
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:33730560
module_fork_in_progress:0
module_fork_last_cow_size:0
aof_current_size:1095799679
aof_base_size:776609932
aof_pending_rewrite:0
aof_buffer_length:62
aof_pending_bio_fsync:0
aof_delayed_fsync:0
# Stats
total_connections_received:100643307
total_commands_processed:563943657
instantaneous_ops_per_sec:201
total_net_input_bytes:99257665731
total_net_output_bytes:13116755001384
total_net_repl_input_bytes:0
total_net_repl_output_bytes:157817942036
instantaneous_input_kbps:10.48
instantaneous_output_kbps:8250.38
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:6.37
rejected_connections:0
sync_full:2
sync_partial_ok:0
sync_partial_err:2
expired_keys:10342045
expired_stale_perc:1.81
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:278448
evicted_keys:0
evicted_clients:0
evicted_scripts:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:264441795
keyspace_misses:7230414
pubsub_channels:1
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:27178
total_forks:102
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:37
dump_payload_sanitizations:0
total_reads_processed:664967825
total_writes_processed:650297847
io_threaded_reads_processed:0
io_threaded_writes_processed:0
io_threaded_freed_objects:0
io_threaded_accept_processed:0
io_threaded_poll_processed:0
io_threaded_total_prefetch_batches:0
io_threaded_total_prefetch_entries:0
client_query_buffer_limit_disconnections:0
client_output_buffer_limit_disconnections:0
reply_buffer_shrinks:3780332
reply_buffer_expands:1693108
eventloop_cycles:736841144
eventloop_duration_sum:29448101737
eventloop_duration_cmd_sum:2888655844
instantaneous_eventloop_cycles_per_sec:286
instantaneous_eventloop_duration_usec:71
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
# Replication
role:master
connected_slaves:2
slave0:ip=valkey-node-1.default-valkey-headless.default.svc.cluster.local,port=6379,state=online,offset=78160824927,lag=0,type=replica
slave1:ip=valkey-node-2.default-valkey-headless.default.svc.cluster.local,port=6379,state=online,offset=78160826359,lag=0,type=replica
replicas_waiting_psync:0
master_failover_state:no-failover
master_replid:0c1b569de90bdada3f0ae6f24b30968609c71c30
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:78160829548
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:10485760
repl_backlog_first_byte_offset:78150337251
repl_backlog_histlen:10492298
# CPU
used_cpu_sys:22139.325850
used_cpu_user:11741.507513
used_cpu_sys_children:88.932730
used_cpu_user_children:626.560572
used_cpu_sys_main_thread:21982.617265
used_cpu_user_main_thread:11731.478573
# Modules
# Commandstats
cmdstat_zrangebyscore:calls=5636,usec=36935,usec_per_call=6.55,rejected_calls=0,failed_calls=0
cmdstat_auth:calls=100549017,usec=306931596,usec_per_call=3.05,rejected_calls=0,failed_calls=0
cmdstat_zrem:calls=36098,usec=74704,usec_per_call=2.07,rejected_calls=0,failed_calls=0
cmdstat_exists:calls=23825035,usec=32026493,usec_per_call=1.34,rejected_calls=0,failed_calls=0
cmdstat_zadd:calls=12335427,usec=117336965,usec_per_call=9.51,rejected_calls=0,failed_calls=0
cmdstat_ping:calls=47332120,usec=25691261,usec_per_call=0.54,rejected_calls=27,failed_calls=0
cmdstat_client|setname:calls=15721,usec=13559,usec_per_call=0.86,rejected_calls=0,failed_calls=0
cmdstat_replconf:calls=1872036,usec=3272735,usec_per_call=1.75,rejected_calls=0,failed_calls=0
cmdstat_del:calls=729471,usec=3890541,usec_per_call=5.33,rejected_calls=0,failed_calls=0
cmdstat_setex:calls=23063240,usec=134444363,usec_per_call=5.83,rejected_calls=0,failed_calls=0
cmdstat_get:calls=60236243,usec=105673010,usec_per_call=1.75,rejected_calls=0,failed_calls=0
cmdstat_subscribe:calls=3,usec=12,usec_per_call=4.00,rejected_calls=0,failed_calls=0
cmdstat_psync:calls=2,usec=56,usec_per_call=28.00,rejected_calls=10,failed_calls=0
cmdstat_select:calls=100156129,usec=42477714,usec_per_call=0.42,rejected_calls=0,failed_calls=0
cmdstat_publish:calls=1387493,usec=13140230,usec_per_call=9.47,rejected_calls=0,failed_calls=0
cmdstat_mget:calls=187604816,usec=2038415126,usec_per_call=10.87,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=15715,usec=2878569,usec_per_call=183.17,rejected_calls=0,failed_calls=0
cmdstat_info:calls=297451,usec=31391678,usec_per_call=105.54,rejected_calls=0,failed_calls=0
cmdstat_set:calls=4450095,usec=30890351,usec_per_call=6.94,rejected_calls=0,failed_calls=0
cmdstat_zrange:calls=479,usec=4431,usec_per_call=9.25,rejected_calls=0,failed_calls=0
cmdstat_slowlog|get:calls=15715,usec=48889,usec_per_call=3.11,rejected_calls=0,failed_calls=0
cmdstat_slowlog|len:calls=15715,usec=16626,usec_per_call=1.06,rejected_calls=0,failed_calls=0
# Errorstats
errorstat_LOADING:count=25
errorstat_NOAUTH:count=12
# Latencystats
latency_percentiles_usec_zrangebyscore:p50=6.015,p99=24.063,p99.9=40.191
latency_percentiles_usec_auth:p50=3.007,p99=9.023,p99.9=31.103
latency_percentiles_usec_zrem:p50=1.003,p99=16.063,p99.9=28.031
latency_percentiles_usec_exists:p50=1.003,p99=5.023,p99.9=19.071
latency_percentiles_usec_zadd:p50=9.023,p99=27.007,p99.9=48.127
latency_percentiles_usec_ping:p50=0.001,p99=2.007,p99.9=11.007
latency_percentiles_usec_client|setname:p50=1.003,p99=3.007,p99.9=15.039
latency_percentiles_usec_replconf:p50=2.007,p99=3.007,p99.9=23.039
latency_percentiles_usec_del:p50=5.023,p99=15.039,p99.9=35.071
latency_percentiles_usec_setex:p50=5.023,p99=18.047,p99.9=38.143
latency_percentiles_usec_get:p50=1.003,p99=7.007,p99.9=26.111
latency_percentiles_usec_subscribe:p50=3.007,p99=6.015,p99.9=6.015
latency_percentiles_usec_psync:p50=28.031,p99=28.031,p99.9=28.031
latency_percentiles_usec_select:p50=0.001,p99=2.007,p99.9=8.031
latency_percentiles_usec_publish:p50=9.023,p99=28.031,p99.9=60.159
latency_percentiles_usec_mget:p50=9.023,p99=39.167,p99.9=64.255
latency_percentiles_usec_config|get:p50=176.127,p99=264.191,p99.9=391.167
latency_percentiles_usec_info:p50=97.279,p99=243.711,p99.9=317.439
latency_percentiles_usec_set:p50=7.007,p99=20.095,p99.9=38.143
latency_percentiles_usec_zrange:p50=7.007,p99=43.007,p99.9=96.255
latency_percentiles_usec_slowlog|get:p50=3.007,p99=11.007,p99.9=27.007
latency_percentiles_usec_slowlog|len:p50=1.003,p99=3.007,p99.9=18.047
# Cluster
cluster_enabled:0
# Keyspace
db1:keys=310576,expires=225298,avg_ttl=299624220
db2:keys=2,expires=0,avg_ttl=3193
db3:keys=1,expires=0,avg_ttl=0
db4:keys=13401,expires=13401,avg_ttl=2713508
db5:keys=13,expires=12,avg_ttl=828195
db6:keys=1,expires=0,avg_ttl=0
db9:keys=8,expires=0,avg_ttl=0
db11:keys=784704,expires=784703,avg_ttl=1320255251
db12:keys=4,expires=4,avg_ttl=137119
------ CLIENT LIST OUTPUT ------
id=98455479 addr=10.30.5.83:41136 laddr=10.30.10.193:6379 fd=36 name=*redacted* age=33927 idle=16 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=1297905 tot-net-out=1008156130 tot-cmds=14378
id=98456794 addr=10.30.14.215:43220 laddr=10.30.10.193:6379 fd=22 name=*redacted* age=33910 idle=43 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=768088 tot-net-out=16623 tot-cmds=2771
id=100643210 addr=10.30.1.134:57410 laddr=10.30.10.193:6379 fd=32 name=*redacted* age=2 idle=2 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=301 tot-net-out=280638 tot-cmds=6
id=96449597 addr=10.30.14.213:58210 laddr=10.30.10.193:6379 fd=62 name=*redacted* age=44934 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=3706881 tot-net-out=40101106 tot-cmds=112330
id=100624524 addr=10.30.14.213:46264 laddr=10.30.10.193:6379 fd=51 name=*redacted* age=346 idle=36 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=545 tot-net-out=561266 tot-cmds=10
id=100624525 addr=10.30.14.213:46276 laddr=10.30.10.193:6379 fd=55 name=*redacted* age=346 idle=36 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=4861 tot-net-out=106 tot-cmds=18
id=99621088 addr=10.30.9.20:60058 laddr=10.30.10.193:6379 fd=46 name=*redacted* age=21137 idle=118 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=31275 tot-net-out=311244 tot-cmds=948
id=100506992 addr=10.30.13.192:50432 laddr=10.30.10.193:6379 fd=27 name=*redacted* age=2005 idle=140 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=223467 tot-net-out=2254420 tot-cmds=6772
id=100506993 addr=10.30.6.63:53120 laddr=10.30.10.193:6379 fd=28 name=*redacted* age=2005 idle=140 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=223467 tot-net-out=2254420 tot-cmds=6772
id=100506994 addr=10.30.4.11:58654 laddr=10.30.10.193:6379 fd=35 name=*redacted* age=2005 idle=140 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=223467 tot-net-out=2254420 tot-cmds=6772
id=100643279 addr=10.30.1.140:47376 laddr=10.30.10.193:6379 fd=45 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=301 tot-net-out=280638 tot-cmds=6
id=100506995 addr=10.30.14.214:56780 laddr=10.30.10.193:6379 fd=50 name=*redacted* age=2005 idle=140 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=223467 tot-net-out=2254420 tot-cmds=6772
id=98693391 addr=10.30.6.50:55208 laddr=10.30.10.193:6379 fd=38 name=*redacted* age=31186 idle=43 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=107043 tot-net-out=891560 tot-cmds=3244
id=98511471 addr=10.30.4.12:56340 laddr=10.30.10.193:6379 fd=37 name=*redacted* age=33336 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=1022554 tot-net-out=10096245 tot-cmds=30975
id=8 addr=10.30.10.193:60256 laddr=10.30.10.193:6379 fd=14 name=*redacted* age=942903 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=277 obl=0 oll=0 omem=0 tot-mem=22200 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=138 tot-net-out=384335065 tot-cmds=3
id=9 addr=10.30.28.244:40732 laddr=10.30.10.193:6379 fd=15 name=*redacted* age=942903 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=23168 events=r cmd=publish user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=142116280 tot-net-out=659336186 tot-cmds=1463944
id=10 addr=10.30.28.244:40736 laddr=10.30.10.193:6379 fd=16 name=*redacted* age=942903 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=277 obl=0 oll=0 omem=0 tot-mem=22200 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=138 tot-net-out=384334788 tot-cmds=3
id=21 addr=10.30.32.25:34762 laddr=10.30.10.193:6379 fd=18 name=*redacted* age=942898 idle=0 flags=S capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=1 omem=20504 tot-mem=22168 events=r cmd=replconf user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=67360749 tot-net-out=78160826669 tot-cmds=935930
id=98458124 addr=10.30.5.81:43246 laddr=10.30.10.193:6379 fd=54 name=*redacted* age=33892 idle=237 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=390923 tot-net-out=8115 tot-cmds=1353
id=100643186 addr=10.30.1.137:46276 laddr=10.30.10.193:6379 fd=24 name=*redacted* age=3 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=606 tot-net-out=631423 tot-cmds=11
id=98455307 addr=10.30.5.81:39958 laddr=10.30.10.193:6379 fd=33 name=*redacted* age=33929 idle=118 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=58467 tot-net-out=582340 tot-cmds=1772
id=100643252 addr=10.30.14.69:56896 laddr=10.30.10.193:6379 fd=21 name=*redacted* age=1 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=362 tot-net-out=350795 tot-cmds=7
id=100643219 addr=10.30.1.134:57422 laddr=10.30.10.193:6379 fd=41 name=*redacted* age=2 idle=1 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=301 tot-net-out=280638 tot-cmds=6
id=100643287 addr=10.30.15.177:53022 laddr=10.30.10.193:6379 fd=19 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=240 tot-net-out=210481 tot-cmds=5
id=100514027 addr=10.30.9.17:40746 laddr=10.30.10.193:6379 fd=34 name=*redacted* age=1848 idle=141 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=221157 tot-net-out=2204310 tot-cmds=6702
id=100514028 addr=10.30.4.9:38190 laddr=10.30.10.193:6379 fd=48 name=*redacted* age=1848 idle=141 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=221157 tot-net-out=2204310 tot-cmds=6702
id=100514029 addr=10.30.4.8:52034 laddr=10.30.10.193:6379 fd=73 name=*redacted* age=1848 idle=141 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=221157 tot-net-out=2204310 tot-cmds=6702
id=100514030 addr=10.30.14.216:59594 laddr=10.30.10.193:6379 fd=75 name=*redacted* age=1848 idle=141 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=221157 tot-net-out=2204310 tot-cmds=6702
id=100638355 addr=10.30.6.51:59220 laddr=10.30.10.193:6379 fd=57 name=*redacted* age=118 idle=118 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=118 tot-net-out=70167 tot-cmds=3
id=100643289 addr=10.30.1.140:47380 laddr=10.30.10.193:6379 fd=39 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=179 tot-net-out=140324 tot-cmds=4
id=100638356 addr=10.30.6.51:59230 laddr=10.30.10.193:6379 fd=61 name=*redacted* age=118 idle=118 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=633 tot-net-out=22 tot-cmds=4
id=100514031 addr=10.30.14.216:59600 laddr=10.30.10.193:6379 fd=76 name=*redacted* age=1848 idle=141 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=83566 tot-net-out=96044943 tot-cmds=1371
id=100514032 addr=10.30.14.216:59602 laddr=10.30.10.193:6379 fd=30 name=*redacted* age=1848 idle=141 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=785475 tot-net-out=16438 tot-cmds=2740
id=100593776 addr=10.30.13.198:50834 laddr=10.30.10.193:6379 fd=20 name=*redacted* age=710 idle=111 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=362 tot-net-out=350795 tot-cmds=7
id=100593779 addr=10.30.13.198:50836 laddr=10.30.10.193:6379 fd=58 name=*redacted* age=710 idle=111 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=2837 tot-net-out=70 tot-cmds=12
id=98511511 addr=10.30.4.10:43512 laddr=10.30.10.193:6379 fd=70 name=*redacted* age=33335 idle=43 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=115095 tot-net-out=958660 tot-cmds=3488
id=98455314 addr=10.30.5.81:39968 laddr=10.30.10.193:6379 fd=47 name=*redacted* age=33929 idle=237 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=41293 tot-net-out=47426142 tot-cmds=678
id=5 addr=10.30.32.25:41624 laddr=10.30.10.193:6379 fd=11 name=*redacted* age=942903 idle=0 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=2048 rbp=1024 obl=0 oll=0 omem=0 tot-mem=23168 events=r cmd=publish user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=142120779 tot-net-out=659336899 tot-cmds=1463946
id=100642107 addr=10.30.15.177:51434 laddr=10.30.10.193:6379 fd=59 name=*redacted* age=27 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=8192 rbp=4096 obl=0 oll=0 omem=0 tot-mem=8832 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=2009 tot-net-out=2245034 tot-cmds=34
id=96442763 addr=10.30.14.223:43860 laddr=10.30.10.193:6379 fd=90 name=*redacted* age=44954 idle=16 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=30650456 tot-net-out=614199 tot-cmds=102367
id=98456556 addr=10.30.5.82:60966 laddr=10.30.10.193:6379 fd=74 name=*redacted* age=33913 idle=118 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=58467 tot-net-out=582340 tot-cmds=1772
id=98684853 addr=10.30.13.195:43178 laddr=10.30.10.193:6379 fd=99 name=*redacted* age=31310 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=973384 tot-net-out=9610505 tot-cmds=29485
id=100643298 addr=10.30.14.69:33664 laddr=10.30.10.193:6379 fd=40 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=240 tot-net-out=210481 tot-cmds=5
id=98693304 addr=10.30.6.51:43200 laddr=10.30.10.193:6379 fd=29 name=*redacted* age=31188 idle=118 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=52131 tot-net-out=519172 tot-cmds=1580
id=20 addr=10.30.28.244:40768 laddr=10.30.10.193:6379 fd=17 name=*redacted* age=942898 idle=0 flags=S capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=1 omem=20504 tot-mem=22168 events=r cmd=replconf user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=67370634 tot-net-out=78160826669 tot-cmds=936070
id=6 addr=10.30.32.25:41634 laddr=10.30.10.193:6379 fd=12 name=*redacted* age=942903 idle=0 flags=P capa= db=0 sub=1 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=277 obl=0 oll=0 omem=0 tot-mem=22200 events=r cmd=subscribe user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=138 tot-net-out=384335342 tot-cmds=3
id=99621061 addr=10.30.6.57:54516 laddr=10.30.10.193:6379 fd=43 name=*redacted* age=21138 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=1005303 tot-net-out=10874944 tot-cmds=30464
id=100514043 addr=10.30.6.63:41208 laddr=10.30.10.193:6379 fd=82 name=*redacted* age=1848 idle=140 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=83261 tot-net-out=95694158 tot-cmds=1366
id=100514044 addr=10.30.6.63:41216 laddr=10.30.10.193:6379 fd=83 name=*redacted* age=1848 idle=140 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=804338 tot-net-out=16378 tot-cmds=2730
id=100643302 addr=10.30.1.137:46324 laddr=10.30.10.193:6379 fd=10 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=240 tot-net-out=210481 tot-cmds=5
id=100643304 addr=10.30.13.186:54608 laddr=10.30.10.193:6379 fd=26 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=301 tot-net-out=280638 tot-cmds=6
id=100643305 addr=10.30.14.219:54806 laddr=10.30.10.193:6379 fd=44 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=118 tot-net-out=70167 tot-cmds=3
id=98455837 addr=10.30.14.215:35764 laddr=10.30.10.193:6379 fd=49 name=*redacted* age=33923 idle=43 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=115953 tot-net-out=965810 tot-cmds=3514
id=98455839 addr=10.30.14.215:35772 laddr=10.30.10.193:6379 fd=53 name=*redacted* age=33923 idle=43 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=84542 tot-net-out=97167455 tot-cmds=1387
id=99619327 addr=10.30.14.222:52988 laddr=10.30.10.193:6379 fd=69 name=*redacted* age=21162 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=603825 tot-net-out=5964506 tot-cmds=18298
id=7 addr=10.30.10.193:60244 laddr=10.30.10.193:6379 fd=13 name=*redacted* age=942903 idle=1 flags=N capa= db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22144 events=r cmd=ping user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=142170903 tot-net-out=659362992 tot-cmds=1464746
id=96442372 addr=10.30.14.223:49336 laddr=10.30.10.193:6379 fd=63 name=*redacted* age=44956 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=3715131 tot-net-out=40190356 tot-cmds=112580
id=98455529 addr=10.30.5.83:41142 laddr=10.30.10.193:6379 fd=25 name=*redacted* age=33926 idle=16 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=8738290 tot-net-out=172371 tot-cmds=28729
id=100643307 addr=10.30.14.219:54814 laddr=10.30.10.193:6379 fd=31 name=*redacted* age=0 idle=0 flags=N capa= db=1 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=109 tot-net-out=615 tot-cmds=3
id=100643308 addr=10.30.32.40:52652 laddr=10.30.10.193:6379 fd=60 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=118 tot-net-out=70167 tot-cmds=3
id=96442376 addr=10.30.14.223:49344 laddr=10.30.10.193:6379 fd=77 name=*redacted* age=44956 idle=16 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=3824960 tot-net-out=3591547361 tot-cmds=51205
id=98456574 addr=10.30.3.225:45880 laddr=10.30.10.193:6379 fd=85 name=*redacted* age=33913 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=1844229 tot-net-out=19950598 tot-cmds=55886
id=100624519 addr=10.30.13.195:47872 laddr=10.30.10.193:6379 fd=81 name=*redacted* age=346 idle=36 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=423 tot-net-out=420952 tot-cmds=8
id=100624520 addr=10.30.13.195:47888 laddr=10.30.10.193:6379 fd=84 name=*redacted* age=346 idle=36 flags=N capa= db=4 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=set user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=3597 tot-net-out=82 tot-cmds=14
id=98685151 addr=10.30.13.198:36884 laddr=10.30.10.193:6379 fd=87 name=*redacted* age=31305 idle=43 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=107043 tot-net-out=891560 tot-cmds=3244
id=98455477 addr=10.30.5.83:41128 laddr=10.30.10.193:6379 fd=56 name=*redacted* age=33927 idle=16 flags=N capa= db=5 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1664 events=r cmd=get user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=1040135 tot-net-out=10265128 tot-cmds=31496
id=100643309 addr=10.30.32.40:52654 laddr=10.30.10.193:6379 fd=52 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=mget user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=118 tot-net-out=70167 tot-cmds=3
id=100643310 addr=10.30.14.67:50748 laddr=10.30.10.193:6379 fd=23 name=*redacted* age=0 idle=0 flags=N capa= db=9 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=17024 events=r cmd=select user=*redacted* redir=-1 resp=2 lib-name= lib-ver= tot-net-in=57 tot-net-out=10 tot-cmds=2
------ MODULES INFO OUTPUT ------
------ CONFIG DEBUG OUTPUT ------
repl-diskless-sync yes
dual-channel-replication-enabled no
sanitize-dump-payload no
lazyfree-lazy-eviction yes
repl-diskless-load disabled
activedefrag no
lazyfree-lazy-user-flush yes
proto-max-bulk-len 512mb
replica-read-only yes
slave-read-only yes
client-query-buffer-limit 1gb
io-threads 1
import-mode no
lazyfree-lazy-server-del yes
debug-context ""
lazyfree-lazy-expire yes
list-compress-depth 0
lazyfree-lazy-user-del yes
------ FAST MEMORY TEST ------
1:M 01 Oct 2025 03:18:36.993 # Bio worker thread #0 terminated
1:M 01 Oct 2025 03:18:36.993 # Bio worker thread #1 terminated
1:M 01 Oct 2025 03:18:36.993 # Bio worker thread #2 terminated
*** Preparing to test memory region 585ed7a08000 (2355200 bytes)
*** Preparing to test memory region 585ed85e0000 (536576 bytes)
*** Preparing to test memory region 7b0247600000 (335544320 bytes)
*** Preparing to test memory region 7b025b800000 (8388608 bytes)
*** Preparing to test memory region 7b025c000000 (135168 bytes)
*** Preparing to test memory region 7b0260600000 (2097152 bytes)
*** Preparing to test memory region 7b0260a00000 (8388608 bytes)
*** Preparing to test memory region 7b0261200000 (2097152 bytes)
*** Preparing to test memory region 7b0261600000 (8388608 bytes)
*** Preparing to test memory region 7b0261e00000 (1746927616 bytes)
*** Preparing to test memory region 7b02ca100000 (13631488 bytes)
*** Preparing to test memory region 7b02caf00000 (9437184 bytes)
*** Preparing to test memory region 7b02cb9fd000 (8388608 bytes)
*** Preparing to test memory region 7b02cc1fe000 (8388608 bytes)
*** Preparing to test memory region 7b02cc9ff000 (8388608 bytes)
*** Preparing to test memory region 7b02cd200000 (8388608 bytes)
*** Preparing to test memory region 7b02cda00000 (8388608 bytes)
*** Preparing to test memory region 7b02ce280000 (20480 bytes)
*** Preparing to test memory region 7b02ce459000 (53248 bytes)
*** Preparing to test memory region 7b02ce8e9000 (12288 bytes)
*** Preparing to test memory region 7b02cea77000 (8192 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------
Symbol: expireScanCallback (base: 0x585ed780fb70)
Module: valkey-server *:6379 (base 0x585ed7645000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x585ed780fb70 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
1:M 01 Oct 2025 03:18:41.504 # dump of function (hexdump of 141 bytes):
554889e541554154534883ec08f64604010f84f9000000488b4610488b57084889fb488b3f4989c44929d44885c00f88dc0000004839d07c274d85e47e084c0163208343280148834310014883c4085b415c415d5dc3662e0f1f840000000000e8cbfdffff85c074d04c8d2d40c220004883431801418b8df005000085c975b9e83b89feffe836e6edff418b95
=== VALKEY BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
https://github.com/valkey-io/valkey/issues
If a module was involved, please open in the module's repo instead.
Suspect RAM error? Use valkey-server --test-memory to verify it.
Some other issues could be detected by valkey-server --check-systemAdditional information
Environment:
- Valkey version: 8.1.3 (based on Redis 7.2.4)
- Mode: Standalone, AOF enabled
- Uptime before crash: ~10 days
- Memory: ~5.5 GB used, fragmentation ratio ~1.01
- Large number of expiring keys: db11 (~785k), db1 (~225k)
- Config includes lazyfree-lazy-expire yesMetadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working