Skip to content

Reduce job update rebuilds and make job creation faster#1582

Open
deep1401 wants to merge 6 commits intomainfrom
fix/jobs-caching
Open

Reduce job update rebuilds and make job creation faster#1582
deep1401 wants to merge 6 commits intomainfrom
fix/jobs-caching

Conversation

@deep1401
Copy link
Member

@deep1401 deep1401 commented Mar 18, 2026

  ┌──────────────────────────────────┬───────────────────┬───────────────────────────────────────────────┐
  │             Trigger              │      Before       │                     After                     │
  ├──────────────────────────────────┼───────────────────┼───────────────────────────────────────────────┤
  │ Job creation                     │ 2–3 full rebuilds │ 0 rebuilds — _add_job() writes index directly │
  ├──────────────────────────────────┼───────────────────┼───────────────────────────────────────────────┤
  │ Status → RUNNING/LAUNCHING       │ 1 full rebuild    │ Nothing                                       │
  ├──────────────────────────────────┼───────────────────┼───────────────────────────────────────────────┤
  │ Status → COMPLETE/STOPPED/FAILED │ 1 full rebuild    │ Inline cached_jobs update                     │
  ├──────────────────────────────────┼───────────────────┼───────────────────────────────────────────────┤
  │ Status → DELETED                 │ 2 full rebuilds   │ Inline index removal                          │
  ├──────────────────────────────────┼───────────────────┼───────────────────────────────────────────────┤
  │ delete_all_jobs()                │ 1 full rebuild    │ Nothing (handled per-job)                     │
  ├──────────────────────────────────┼───────────────────┼───────────────────────────────────────────────┤
  │ Lab facade per job run           │ 3–4 rebuilds      │ 0 rebuilds                                    │
  └──────────────────────────────────┴───────────────────┴───────────────────────────────────────────────┘

@deep1401 deep1401 changed the title Make less updates to jobs index and dont do cached jobs anymore Make less updates to jobs index and dont do batched cached job updates Mar 18, 2026
@deep1401 deep1401 changed the title Make less updates to jobs index and dont do batched cached job updates Reduce job update rebuilds and make job creation faster Mar 18, 2026
@deep1401 deep1401 marked this pull request as ready for review March 18, 2026 20:47
@sentry
Copy link

sentry bot commented Mar 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant