lookupCommandByCString in VM_Replicate

[xingbowang]

The problem/use-case that the feature addresses

Avoid performance hit in VM_Replicate function call.

Description of the feature

My team recently was trying to optimize the performance of our module. We found this function call lookupCommandByCString inside VM_Replicate to taking a lot of CPU time.
https://github.com/valkey-io/valkey/blob/unstable/src/module.c#L3539-L3540

The main reason is that it allocates memory to convert const char * into sds, then perform a dict lookup. From VM_Replicate function, it looks like it is just doing a sanity check without using it for anything else. My guess is the purpose is to protect the engine from running a 3rd party module. However, the protection seems induces too much performance penalty. If we develop our in-house module and don't run other modules, could we bypass this check to avoid performance hit. Is it possible to add a config flag "trust-module-replication" to bypass this check?

Alternatives you've considered

We considered VM_ReplicateVerbatim, but we sometimes needs to rewrite the command during replication. So we could not use it.

Additional information

Any additional information that is relevant to the feature request.

[hpatro]

This helps avoiding unnecessary check/cpu cycles while using modules trusted by the system. @valkey-io/core-team If there is no major concern we can implement this.

[madolson]

I'm fine with it. We could make it so that if cmdname is NULL it bypasses the check and just replicates whatever is in the va_args. I'm fine with a new module API as well.

[zuiderkwast]

A new flag to skip the command lookup sounds good to me.

"S" for skip? Or "B" for bypass?

I see we even create this sds twice: first for the command lookup and then again when creating the argv for replication. If we move the command lookup from VM_Replicate to moduleCreateArgvFromUserFormat (where we check the flags), then we can also do the command lookup by sds string after creating the sds string which we need to do anyway for replication, so we don't need to create the same sds twice.

[madolson]

A new flag to skip the command lookup sounds good to me.

This wasn't the original ask, the ask was to add a server config. I don't think we should do that. I think we should add a separate module API that accepts flags. I don't think the Flags should be single characters, I think they should be an enum. I would prefer the term SKIP_VALIDATION.

[hwware]

Sorry, do you mean the function lookupCommandBySdsLogic?

valkey/src/server.c

Line 3152 in 285064b

struct serverCommand *lookupCommandBySdsLogic(dict *commands, sds s) {

because I do not find any codes of converting const char * into sds

[hwware]

Is there to update the API struct serverCommand *lookupCommandBySdsLogic(dict commands, sds s), update sds s to char part instead of create a new flag？
If i understand correctly, you think the most cost of CPU is line

valkey/src/server.c

Line 3156 in 285064b

if (argc < 1 || argc > 2) {

[xingbowang]

The function is lookupCommandByCString, which call lookupCommandByCStringLogic, which call lookupCommandBySdsLogic, during which a temp sds is created. The memory allocation and free is the main overhead.

[zuiderkwast]

@xingbowang If this temp sds allocation and free is the main overhead, then it can be solved by the optimization: move the command lookup to moduleCreateArgvFromUserFormat after creating the argv and use lookupCommandBySds. No new config required. Correct?

[xingbowang]

@xingbowang If this temp sds allocation and free is the main overhead, then it can be solved by the optimization: move the command lookup to moduleCreateArgvFromUserFormat after creating the argv and use lookupCommandBySds. No new config required. Correct?

I don't think it solves the problem. moduleCreateArgvFromUserFormat is called during VM_Call, which is when the module command is executed. However, the issue happens inside VM_Replicate. E.g. getset was replicate as set. So lookup the command at VM_call would not help, as the command in VM_replicate could be different.

Meantime, I don't know why the code lookup the command inside VM_Replicate. Because it was not used in anyway after the lookup. I talked with Madelyn, who suggested that it was used mainly for debugging purpose. I guess the reason could be replicated command name could be different from the command get called. So developer need to validate it. THis makes more sense to turn on/off as a debugging feature.

[sungming2]

I started working on this issue

[sungming2]

@madolson This wasn't the original ask, the ask was to add a server config. I don't think we should do that. I think we should add a separate module API that accepts flags. I don't think the Flags should be single characters, I think they should be an enum. I would prefer the term SKIP_VALIDATION.

I have a few questions on this.

1. Can I know why we should not add a new flag to format (fmt)? since the lookupCommands is just debugging/checking purpose in the replication, I think we could use the flag to bypass it after setting up the all flags.
2. Could you elaborate on "add a separate module API accepting flags"? Does it mean we want to create another new API taking the new flag like VM_ReplicateWithFlags?

[hpatro]

I have a few questions on this.

1. Can I know why we should not add a new flag to format (fmt)? since the lookupCommands is just debugging/checking purpose in the replication, I think we could use the flag to bypass it after setting up the all flags.

To avoid mixing the utility of format specifier (command behavior) vs flags where we could indicate to bypass validation check.

2. Could you elaborate on "add a separate module API accepting flags"? Does it mean we want to create another new API taking the new flag like VM_ReplicateWithFlags?

Yes. This seems more cleaner to me as well.