Skip to content

Improve storage manager and merge it with creation_management module #147

@vdusek

Description

@vdusek

The current Crawlee / StorageClientManager is more or less just copied from the Python SDK / StorageClientManager and is extremely simple. Its primary role is to maintain and provide access to storage client instances based on specific input parameters.

The Crawlee TS / StorageManager is more complex and it takes care of more things - creating instances of storages & their caching.

Currently, we have a helper module "creation_management" in storages/ which helps with it.

Let's move logic from storages/creation_management to StorageClientManager and improve the creation & caching process.

Functions get_or_create, find_or_create_client_by_id_or_name a create_*_from_directory should be refactored.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions