Skip to content

replicate generator pickle support improvements#1190

Merged
jmartin-tech merged 1 commit intoNVIDIA:mainfrom
jmartin-tech:fix/replicate-multiprocessing
May 2, 2025
Merged

replicate generator pickle support improvements#1190
jmartin-tech merged 1 commit intoNVIDIA:mainfrom
jmartin-tech:fix/replicate-multiprocessing

Conversation

@jmartin-tech
Copy link
Collaborator

fix #1160

Following patterns established in openai add _load and _clear methods for client objects. A number of other generators have be identified for potential enhancement for objects that cannot pickle in generators. These additional generators will be addressed separately.

Verification

List the steps needed to make sure this thing works

  • Test multiprocess generation
python3 -m garak -m replicate -n deepseek-ai/deepseek-v3 --p lmrc --parallel_requests 2 -g 3
  • CI automation additions pass

Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
@jmartin-tech jmartin-tech force-pushed the fix/replicate-multiprocessing branch from 1fe7d39 to 719a089 Compare April 30, 2025 22:13
Copy link
Collaborator

@erickgalinkin erickgalinkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Shout out for the added test!

@jmartin-tech jmartin-tech merged commit da55233 into NVIDIA:main May 2, 2025
9 checks passed
@jmartin-tech jmartin-tech deleted the fix/replicate-multiprocessing branch May 2, 2025 16:50
@github-actions github-actions bot locked and limited conversation to collaborators May 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replicate generator error 'TypeError: cannot pickle 'module' object' when parallel requests or parallel attempts > 1

2 participants