Skip to content

Conversation

@philandstuff
Copy link
Contributor

@philandstuff philandstuff commented May 19, 2025

This change allows a predictor input to be defined exclusively as **kwargs. This is an experimental change that allows a model to be used as a proxy where any inputs will be passed through directly without modification.

import json

def predict(**kwargs) -> str:
  return json.dumps(kwargs)

This change restricts the use of **kwargs to functions/methods that accept only that as an argument to avoid the complexity of having to support both dynamic and fixed inputs.

The OpenAPI schema for such a model will be:

{
   "properties": {},
   "additionalProperties": true,
   "type": "object",
   "title": "Input"
}

philandstuff and others added 7 commits May 19, 2025 11:25
This is an experimental change to support **kwargs on predictors.  You can
annotate them as normal and the OpenAPI schema will reflect the type of **kwargs
via additionalProperties.
Also to use `Union` rather than the pipe symbol which is not supported
on older Python versions.
This unlocks our use case of using a cog model as a proxy without making
the interface more complex by supporting partially defined inputs
alongside the `**kwargs` parameter.
@aron
Copy link
Contributor

aron commented May 20, 2025

@philandstuff I've reworked this PR to be a bit more restrictive. We now only allow models to use the **kwargs syntax as the sole argument. We also ignore the typing. This allows a model to receive arbitrary inputs but cog won't do any processing on these (such as downloading file inputs).

I think this is a good enough start for our use case.

@aron aron requested a review from Copilot May 21, 2025 13:57
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces experimental support for predictors that exclusively use **kwargs as input, enabling dynamic passthrough of input parameters for proxy models. Key changes include:

  • Adding tests to verify kwargs input behavior.
  • Implementing a new code branch in get_input_create_model_kwargs to support functions with only **kwargs.
  • Updating the OpenAPI schema and adjusting create_model kwargs configuration.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
python/tests/server/test_http_input.py Added test to validate dynamic kwargs inputs in predictions
python/tests/server/fixtures/input_kwargs.py New predictor implementation accepting only **kwargs
python/cog/predictor.py Updated input model generation logic to support exclusive **kwargs
.tool-versions Updated Python version to 3.13.2
Comments suppressed due to low confidence (1)

python/tests/server/test_http_input.py:45

  • [nitpick] The variable name 'input' shadows the built-in function. Consider renaming it to 'input_data' to improve clarity.
input = {"animal": "giraffe", "no": 5}

if order != 0:
raise TypeError(f"Unsupported keyword parameter **{name}")

class ExtraKeywordInput(BaseInput):
Copy link

Copilot AI May 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider adding a comment explaining the purpose of the 'ExtraKeywordInput' class to clarify its role in handling exclusive **kwargs inputs.

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

@philandstuff philandstuff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aron thanks for the polish. LGTM

aron and others added 2 commits May 21, 2025 15:10
Co-authored-by: Philip Potter <[email protected]>
Signed-off-by: Aron Carroll <[email protected]>
Co-authored-by: Philip Potter <[email protected]>
Signed-off-by: Aron Carroll <[email protected]>
@aron aron enabled auto-merge (squash) May 21, 2025 14:10
@aron aron merged commit 0f4cf3d into main May 21, 2025
21 checks passed
@aron aron deleted the wip-support-kwargs-predictors branch May 21, 2025 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants