Improve get dag grid structure endpoint speed #55942

dor-bernstein · 2025-09-22T09:55:34Z

closes: #55941
related: #55941

Seems like the grid structure endpoint doesn't filter on dag_id when querying for previous dag versions.
Added this filter and ran it on my prod_db. The query went from 2 minutes to ~1 second.

boring-cyborg · 2025-09-22T09:55:38Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: [email protected]
Slack: https://s.apache.org/airflow-slack

jason810496

Nice catch!

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py

bugraoz93 · 2025-09-22T20:40:22Z

Thanks for the PR! You can resolve comments that have been updated

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py

eladkal · 2025-09-23T12:22:53Z

Merging. Failures are not related to this PR

boring-cyborg · 2025-09-23T12:23:22Z

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

pierrejeambrun

Nice catch! Thanks

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

hussein-awala · 2025-09-30T15:47:10Z

The SerializedDagModel.id != latest_serdag.id filter was misplaced in the original query, and it's the reason of the performance issue, I tested this PR without your filter and it solved the issue, but I kept your filter to avoid re-introducing the bug.

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

* Improve get dag structure endpoint speed --------- (cherry picked from commit 311277c) Co-authored-by: dor-bernstein <[email protected]> Co-authored-by: Elad Kalif <[email protected]>

…che#55942)

* Improve get dag structure endpoint speed --------- (cherry picked from commit 311277c) Co-authored-by: dor-bernstein <[email protected]> Co-authored-by: Elad Kalif <[email protected]>

…ed (apache#55942)

…56937) * [v3-1-test] Improve get dag grid structure endpoint speed (#55942) * Improve get dag structure endpoint speed --------- (cherry picked from commit 311277c) Co-authored-by: dor-bernstein <[email protected]> Co-authored-by: Elad Kalif <[email protected]> * fixup! fixup! [v3-1-test] Improve get dag grid structure endpoint speed (#55942) --------- Co-authored-by: dor-bernstein <[email protected]> Co-authored-by: Elad Kalif <[email protected]>

dheerajturaga · 2025-10-22T15:39:49Z

I created this #56968 dint realize it was already fixed!

Thanks for fixing! this has been bothering me a lot!

imporve get dag structure endpoint speed

31ef3fe

dor-bernstein requested review from bugraoz93, ephraimbuddy, jason810496, pierrejeambrun, rawwar and shubhamraj-git as code owners September 22, 2025 09:55

boring-cyborg bot added the area:API Airflow's REST/HTTP API label Sep 22, 2025

eladkal added this to the Airflow 3.1.1 milestone Sep 22, 2025

eladkal added the type:bug-fix Changelog: Bug Fixes label Sep 22, 2025

fix import order

5578eef

jason810496 approved these changes Sep 22, 2025

View reviewed changes

ashb reviewed Sep 22, 2025

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py Outdated Show resolved Hide resolved

ashb reviewed Sep 22, 2025

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py Outdated Show resolved Hide resolved

dor-bernstein added 2 commits September 22, 2025 23:30

cr fixes

c61a46d

Merge branch 'main' into issue-55941/grid_api_improvements

5d239af

dor-bernstein requested a review from ashb September 22, 2025 20:31

bugraoz93 approved these changes Sep 22, 2025

View reviewed changes

eladkal reviewed Sep 23, 2025

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py Outdated Show resolved Hide resolved

fix static checks

d216126

eladkal approved these changes Sep 23, 2025

View reviewed changes

eladkal merged commit 311277c into apache:main Sep 23, 2025
106 of 107 checks passed

pierrejeambrun reviewed Sep 24, 2025

View reviewed changes

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Sep 30, 2025

Improve get dag grid structure endpoint speed (apache#55942)

ee18192

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

hussein-awala mentioned this pull request Sep 30, 2025

[WEBSERVER] SerializedDagModel query bug fix impacting Grid view performance #56257

Merged

hussein-awala mentioned this pull request Sep 30, 2025

Dags list is not loading in the UI, espically when filtering with tags #56219

Open

2 tasks

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 1, 2025

Improve get dag grid structure endpoint speed (apache#55942)

c0eced7

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 2, 2025

Improve get dag grid structure endpoint speed (apache#55942)

fdef77f

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 3, 2025

Improve get dag grid structure endpoint speed (apache#55942)

a15f8a1

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 4, 2025

Improve get dag grid structure endpoint speed (apache#55942)

be541ab

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 5, 2025

Improve get dag grid structure endpoint speed (apache#55942)

e71b565

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 5, 2025

Improve get dag grid structure endpoint speed (apache#55942)

ae4aadf

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 7, 2025

Improve get dag grid structure endpoint speed (apache#55942)

38fb8bc

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 8, 2025

Improve get dag grid structure endpoint speed (apache#55942)

d3d1e52

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 9, 2025

Improve get dag grid structure endpoint speed (apache#55942)

a5566df

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 10, 2025

Improve get dag grid structure endpoint speed (apache#55942)

45627b9

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 11, 2025

Improve get dag grid structure endpoint speed (apache#55942)

c5787bd

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 12, 2025

Improve get dag grid structure endpoint speed (apache#55942)

22ff162

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 14, 2025

Improve get dag grid structure endpoint speed (apache#55942)

1ec069b

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 15, 2025

Improve get dag grid structure endpoint speed (apache#55942)

34cb3dd

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 17, 2025

Improve get dag grid structure endpoint speed (apache#55942)

40cdeee

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 19, 2025

Improve get dag grid structure endpoint speed (apache#55942)

b3bb693

* Improve get dag structure endpoint speed --------- Co-authored-by: Elad Kalif <[email protected]>

Lee-W added a commit to astronomer/airflow that referenced this pull request Oct 21, 2025

fixup! [v3-1-test] Improve get dag grid structure endpoint speed (apa…

29f9e1b

…che#55942)

Lee-W added a commit to astronomer/airflow that referenced this pull request Oct 21, 2025

fixup! fixup! [v3-1-test] Improve get dag grid structure endpoint spe…

2985ebd

…ed (apache#55942)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve get dag grid structure endpoint speed #55942

Improve get dag grid structure endpoint speed #55942

Uh oh!

dor-bernstein commented Sep 22, 2025 •

edited

Loading

Uh oh!

boring-cyborg bot commented Sep 22, 2025

Uh oh!

jason810496 left a comment

Uh oh!

Uh oh!

Uh oh!

bugraoz93 commented Sep 22, 2025 •

edited

Loading

Uh oh!

Uh oh!

eladkal commented Sep 23, 2025

Uh oh!

Uh oh!

boring-cyborg bot commented Sep 23, 2025

Uh oh!

pierrejeambrun left a comment

Uh oh!

hussein-awala commented Sep 30, 2025

Uh oh!

dheerajturaga commented Oct 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Improve get dag grid structure endpoint speed #55942

Improve get dag grid structure endpoint speed #55942

Uh oh!

Conversation

dor-bernstein commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boring-cyborg bot commented Sep 22, 2025

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bugraoz93 commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

eladkal commented Sep 23, 2025

Uh oh!

Uh oh!

boring-cyborg bot commented Sep 23, 2025

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

hussein-awala commented Sep 30, 2025

Uh oh!

dheerajturaga commented Oct 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

dor-bernstein commented Sep 22, 2025 •

edited

Loading

bugraoz93 commented Sep 22, 2025 •

edited

Loading