Skip to content

Conversation

@mariosasko
Copy link
Collaborator

Override Pickler.save to implement deterministic reduction (lazily registered; inspired by https://github.com/uqfoundation/dill/blob/master/dill/_dill.py#L343) functions for torch.Tensor and spaCy models.

Fix #5170, fix #3178

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Nov 2, 2022

The documentation is not available anymore as the PR was closed or merged.

@mariosasko mariosasko requested a review from lhoestq November 2, 2022 14:36
@mariosasko mariosasko marked this pull request as ready for review November 2, 2022 14:36
Copy link
Member

@lhoestq lhoestq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it thanks ! Added some minor comments, but overall LGTM :)

@mariosasko mariosasko merged commit 0d9c12a into main Nov 2, 2022
@mariosasko mariosasko deleted the improve-caching branch November 2, 2022 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Caching] Deterministic hashing of torch tensors "Property couldn't be hashed properly" even though fully picklable

4 participants