Skip to content

Conversation

@sarthakaggarwal97
Copy link
Contributor

@sarthakaggarwal97 sarthakaggarwal97 commented Dec 6, 2024

Adds filter options to CLIENT LIST:

* USER <username>
  Return clients authenticated by <username>.
* ADDR <ip:port>
  Return clients connected from the specified address.
* LADDR <ip:port>
  Return clients connected to the specified local address.
* SKIPME (YES|NO)
  Exclude the current client from the list (default: no).
* MAXAGE <maxage>
  Only list connections older than the specified age.

Modifies the ID filter to CLIENT KILL to allow multiple IDs

* ID <client-id> [<client-id>...]
  Kill connections by client ids.

This makes CLIENT LIST and CLIENT KILL accept the same options.

For backward compatibility, the default value for SKIPME is NO for CLIENT LIST and YES for CLIENT KILL.

The MAXAGE comes from CLIENT KILL, where it keeps clients with the given max age and kills the older ones. This logic becomes weird for CLIENT LIST, but is kept for similary with CLIENT KILL, for the use case of first testing manually using CLIENT LIST, and then running CLIENT KILL with the same filters.

The ID client-id [client-id ...] no longer needs to be the last filter. The parsing logic determines if an argument is an ID or not based on whether it can be parsed as an integer or not.

Partly addresses: #668

@codecov
Copy link

codecov bot commented Dec 6, 2024

Codecov Report

Attention: Patch coverage is 91.83673% with 28 lines in your changes missing coverage. Please review.

Project coverage is 70.93%. Comparing base (d13aad4) to head (110c7d3).
Report is 5 commits behind head on unstable.

Files with missing lines Patch % Lines
src/networking.c 91.83% 28 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #1401      +/-   ##
============================================
- Coverage     70.94%   70.93%   -0.01%     
============================================
  Files           120      120              
  Lines         65044    65078      +34     
============================================
+ Hits          46143    46161      +18     
- Misses        18901    18917      +16     
Files with missing lines Coverage Δ
src/commands.def 100.00% <ø> (ø)
src/server.h 100.00% <ø> (ø)
src/networking.c 88.76% <91.83%> (+0.28%) ⬆️

... and 14 files with indirect coverage changes

@sarthakaggarwal97 sarthakaggarwal97 marked this pull request as ready for review December 9, 2024 00:59
Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

I just took a quick look and I can see some things that appear to be broken:

  • CLIENT LIST with multiple IDs, for example CLIENT LIST ID 3 4 5. (It means id is 3 or 4 or 5.) We a test case for this.
  • The default for "skipme" for CLIENT KILL should be YES by default (when filters are used), but for CLIENT LIST, skipme should be NO by default. Otherwise, we have a breaking change. Please add some tests without SKIPME but with other filters matching the calling client to cover the default skipme value for LIST and KILL.

We need some test cases combining multiple filter rules at the same time.

@sarthakaggarwal97
Copy link
Contributor Author

Thanks @zuiderkwast for the comments! I am working on them

@sarthakaggarwal97 sarthakaggarwal97 force-pushed the client-command-revamp branch 2 times, most recently from 049f2d4 to e3c1c5a Compare December 11, 2024 09:23
@sarthakaggarwal97
Copy link
Contributor Author

sarthakaggarwal97 commented Dec 11, 2024

@zuiderkwast thank you for your comments. Adressed most of them. Working on adding tests meanwhile. Trying to see how can I have tests with multiple clients.

Sharing some responses from local testing

Client Kill Command

127.0.0.1:6379> client list
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=11 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=53 tot-net-out=209291 tot-cmds=1
id=4 addr=127.0.0.1:63141 laddr=127.0.0.1:6379 fd=9 name= age=8 idle=8 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
id=5 addr=127.0.0.1:63142 laddr=127.0.0.1:6379 fd=10 name= age=5 idle=5 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client info
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=16 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|info user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=79 tot-net-out=210285 tot-cmds=2
127.0.0.1:6379> client kill id 4
(integer) 1
127.0.0.1:6379> client list
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=39 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=146 tot-net-out=210626 tot-cmds=4
id=5 addr=127.0.0.1:63142 laddr=127.0.0.1:6379 fd=10 name= age=33 idle=33 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client list
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=50 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=172 tot-net-out=211295 tot-cmds=5
id=5 addr=127.0.0.1:63142 laddr=127.0.0.1:6379 fd=10 name= age=44 idle=44 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
id=7 addr=127.0.0.1:63168 laddr=127.0.0.1:6379 fd=9 name= age=3 idle=3 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client kill id 5 7 
(integer) 2
127.0.0.1:6379> client list
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=72 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=246 tot-net-out=212296 tot-cmds=7
id=8 addr=127.0.0.1:63183 laddr=127.0.0.1:6379 fd=9 name= age=7 idle=7 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
id=9 addr=127.0.0.1:63184 laddr=127.0.0.1:6379 fd=10 name= age=4 idle=4 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client kill id 3 9 addr 127.0.0.1:63183
(integer) 0
127.0.0.1:6379> client list
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=97 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=352 tot-net-out=213295 tot-cmds=9
id=8 addr=127.0.0.1:63183 laddr=127.0.0.1:6379 fd=9 name= age=32 idle=32 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
id=9 addr=127.0.0.1:63184 laddr=127.0.0.1:6379 fd=10 name= age=29 idle=29 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client kill id 3 9 addr 127.0.0.1:63184
(integer) 1
127.0.0.1:6379> client list
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=107 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=458 tot-net-out=214298 tot-cmds=11
id=8 addr=127.0.0.1:63183 laddr=127.0.0.1:6379 fd=9 name= age=42 idle=42 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1

Client List Command

127.0.0.1:6379> client list
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=1361 idle=8 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=484 tot-net-out=215973 tot-cmds=13
id=8 addr=127.0.0.1:63183 laddr=127.0.0.1:6379 fd=9 name= age=1296 idle=1296 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
id=10 addr=127.0.0.1:55765 laddr=127.0.0.1:6379 fd=10 name= age=10 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=10 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1946 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=53 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client list id 3 8
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=1368 idle=15 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=484 tot-net-out=215973 tot-cmds=13
id=8 addr=127.0.0.1:63183 laddr=127.0.0.1:6379 fd=9 name= age=1303 idle=1303 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client list id 3 8 age 1300
ERR syntax error
127.0.0.1:6379> client list id 3 8 maxage 1300
id=3 addr=127.0.0.1:63139 laddr=127.0.0.1:6379 fd=8 name= age=1387 idle=34 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=client|list user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=484 tot-net-out=215973 tot-cmds=13
id=8 addr=127.0.0.1:63183 laddr=127.0.0.1:6379 fd=9 name= age=1322 idle=1322 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1
127.0.0.1:6379> client list id 3 8 maxage 1500
127.0.0.1:6379> client list id 3 8 addr 127.0.0.1:63183
id=8 addr=127.0.0.1:63183 laddr=127.0.0.1:6379 fd=9 name= age=1349 idle=1349 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 watch=0 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1920 events=r cmd=command|docs user=default redir=-1 resp=2 lib-name= lib-ver= tot-net-in=27 tot-net-out=209291 tot-cmds=1

Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to see how can I have tests with multiple clients.

You can create multiple clients.

        set c1 [valkey_client]
        set c2 [valkey_client]
        set c3 [valkey_client]
        ...
        $c1 close
        $c2 close
        $c3 close

@sarthakaggarwal97 sarthakaggarwal97 force-pushed the client-command-revamp branch 3 times, most recently from fd743fe to 6681e47 Compare December 16, 2024 21:00
Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The refactoring looks good. I have just a few small comments and there is some merge conflict.

Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just two nits.

@zuiderkwast zuiderkwast added release-notes This issue should get a line item in the release notes major-decision-pending Major decision pending by TSC team needs-doc-pr This change needs to update a documentation page. Remove label once doc PR is open. labels Dec 20, 2024
@zuiderkwast
Copy link
Contributor

@valkey-io/core-team Please approve the extended syntax for CLIENT LIST as described in the top comment.

The MAXAGE argument for CLIENT LIST is a little strange, because it's the max age of clients to exclude, for symmetry with CLIENT KILL. An alternative is to forbid MAXAGE for CLIENT LIST.

@sarthakaggarwal97
Copy link
Contributor Author

sarthakaggarwal97 commented Jan 14, 2025

Thanks folks for the suggestions and inputs.

Based on @hwware suggestions, I performed the SET/GET operations while running client list <filter> <filter-value> for 10000 clients.
I do not see any major deviation in performance from the current unstable command.

Sharing the results:

Screenshot 2025-01-14 at 3 09 17 PM

Notes:

  1. Latency Unit: msec
  2. RPS: requests per second

@hpatro
Copy link
Collaborator

hpatro commented Jan 14, 2025

@sarthakaggarwal97 Thanks for sharing the results and is inline with our expectation. I'm good with the stats. The above benchmarking is performed with one client running the CLIENT LIST (filters) and 10k clients running SET/GET operations. The setup is quite similar to a real world scenario where the no. of application clients >>> no. of administrator clients.

@hwware
Copy link
Member

hwware commented Jan 15, 2025

It looks good to me. Approved

@hpatro
Copy link
Collaborator

hpatro commented Jan 15, 2025

I see atleast 4 core-team member have approved this either through voting or PR approval. @valkey-io/core-team Do you want to review further or is it good to be merged?

@zuiderkwast zuiderkwast added major-decision-approved Major decision approved by TSC team and removed major-decision-pending Major decision pending by TSC team labels Jan 15, 2025
@zuiderkwast
Copy link
Contributor

Let's merge it.

@sarthakaggarwal97 do you want to update the docs too? The commands/*.md files in the valkey-doc repo. We can have one doc PR for all these updates if you want.

@zuiderkwast zuiderkwast merged commit 6a8f068 into valkey-io:unstable Jan 15, 2025
51 checks passed
@sarthakaggarwal97
Copy link
Contributor Author

Thank you everyone for helping in this one.

@zuiderkwast let me raise the doc PR as well. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

major-decision-approved Major decision approved by TSC team needs-doc-pr This change needs to update a documentation page. Remove label once doc PR is open. release-notes This issue should get a line item in the release notes

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

8 participants