Skip to content

Task result storage and retrieval #166

@chrisguidry

Description

@chrisguidry

So far, Docket's design has been opinionated about tasks being one-way/fire-and-forget. However, a common expectation of background task systems is that they can allow for task result storage and retrieval.

Let's create a great DX for result storage that doesn't create overhead for one-way tasks. We can do this with the return type annotation of None versus a cloudpickle-serializable type. We can also explore more of an opt-in approach with a task dependency for manipulating storage (as opposed to having the Worker do it.

Requirements:

  • Results storage, with redis as a default implementation, but perhaps pluggable for different backends
  • Unification of task exceptions with result storage
  • No performance penalty for tasks that are one-way operations (i.e. -> None)
  • A way to set the expiration time for results, either per-task or globally
  • A way to pick up the results of a task by its task key
  • A way to distinguish between "this task returned None" and "this task hasn't run yet"

This may tie in somehow with #88 as well.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions