Skip to content

feat: allow passing serializer on cache creation and implement JSONSerializer#462

Open
john0isaac wants to merge 1 commit into
mainfrom
feat/extensible-serializers
Open

feat: allow passing serializer on cache creation and implement JSONSerializer#462
john0isaac wants to merge 1 commit into
mainfrom
feat/extensible-serializers

Conversation

@john0isaac
Copy link
Copy Markdown
Collaborator

Allows people to pass the serializers when creating an instance of the class while maintaining a default to pickle serializer if no value is passed.

Adds an implementation of a JSON serializer that only handles the types handled by the json library

closes #136 and related to pallets-eco/flask-caching#209
It should simplify the implementation of pallets-eco/flask-caching#209 as backends will accept them as inputs now without rewriting the backend.

@john0isaac john0isaac self-assigned this May 16, 2026
@john0isaac john0isaac force-pushed the feat/extensible-serializers branch from f5aa5ad to 7f65e69 Compare May 16, 2026 21:02
Comment thread src/cachelib/dynamodb.py
cache_item = self._get_item(self.key_prefix + key)
if cache_item:
response = cache_item[RESPONSE_FIELD]
value = self.serializer.loads(response)
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the JSON serializer will be broken for dynamodb since this unpacking was happening inside the serializer .loads function I moved it here since it makes more sense and will allow other serializers to be used without special handling.

This change doesn't affect end user in anything while allowing for using other serializers.

Comment thread src/cachelib/memcached.py
Comment on lines +32 to +33
This cache doesn't have a serializer since the underlying memcached client
libraries handle serialization internally."
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could implement this but it's not simple so I left it as is.

Comment thread src/cachelib/mongodb.py
Comment on lines +42 to +43
except ImportError as err:
raise RuntimeError("no pymongo module found") from err
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the only cache backend that doesn't raise the error like this.

Also pymongo was unbound when used after these lines since on error we log and continue instead of raising.

Comment thread src/cachelib/mongodb.py

if client is None or isinstance(client, str):
client = pymongo.MongoClient(host=client)
client = pymongo.MongoClient(host=client, **kwargs)
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

**kwargs was there but not used for some reason.

logging.warning(
f"An exception has been raised during a pickling operation: {e}"
)
def _warn(self, e: Exception) -> None:
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

broaden exception to allow for pickle and json errors handling without getting type errors

Comment on lines +124 to +127
class MongoDbSerializer(BaseSerializer):
"""Default serializer for MongoDbCache."""

pass
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was not implemented for some reason

Comment thread src/cachelib/uwsgi.py
Comment on lines +66 to +72
) -> _t.Optional[_t.Literal[True]]:
result = self._uwsgi.cache_update(
key,
self.serializer.dumps(value),
self._normalize_timeout(timeout),
self.cache,
) # type: bool
) # type: _t.Optional[_t.Literal[True]]
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into the code of uWSGI lib. the type is _TrueOrNone which is a type alias to Literal[True] | None

@john0isaac john0isaac force-pushed the feat/extensible-serializers branch from 7f65e69 to dd74f05 Compare May 16, 2026 21:13
@john0isaac john0isaac requested a review from davidism May 16, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

Allow passing serializer as a class parameter.

1 participant