Skip to content

[ty] Shrink reachability constraints #19410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 18, 2025
Merged

[ty] Shrink reachability constraints #19410

merged 1 commit into from
Jul 18, 2025

Conversation

@MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Jul 17, 2025

Summary

Shrink the reachability constraints before storing them in the UseDefMap.
This reduces memory usage on a large project for UseDefMaps by about 400mb or around 8%

Test Plan

cargo test

@MichaReiser MichaReiser added the ty Multi-file analysis & type inference label Jul 17, 2025
@MichaReiser MichaReiser marked this pull request as ready for review July 17, 2025 19:49
@github-actions
Copy link
Contributor

github-actions bot commented Jul 17, 2025

mypy_primer results

No ecosystem changes detected ✅

Memory usage changes were detected when running on open source projects 
flake8 (https://github.com/pycqa/flake8)
-     memo fields = ~66MB
+     memo fields = ~63MB

trio (https://github.com/python-trio/trio)
-     memo fields = ~159MB
+     memo fields = ~152MB

sphinx (https://github.com/sphinx-doc/sphinx)
- TOTAL MEMORY USAGE: ~332MB
+ TOTAL MEMORY USAGE: ~316MB

prefect (https://github.com/PrefectHQ/prefect)
- TOTAL MEMORY USAGE: ~690MB
+ TOTAL MEMORY USAGE: ~657MB

@dcreager dcreager mentioned this pull request Jul 17, 2025
Merged
@MichaReiser MichaReiser merged commit 1765014 into main Jul 18, 2025
38 checks passed
@MichaReiser MichaReiser deleted the micha/shrink-reachability-constraints branch July 18, 2025 05:36
dcreager added a commit that referenced this pull request Jul 18, 2025
@dcreager
Merge branch 'main' into dcreager/smoosh-reachability
31934ad 
* main:
  [ty] Use `…` as the "cut" indicator in diagnostic rendering (#19420)
  [ty] synthesize __setattr__ for frozen dataclasses (#19307)
  [ty] Fixed bug in semantic token provider for parameters. (#19418)
  [ty] Shrink reachability constraints (#19410)
  Move JUnit rendering to `ruff_db` (#19370)
dcreager added a commit that referenced this pull request Jul 21, 2025
@dcreager
[ty] Garbage-collect reachability constraints (#19414)
88de572 
This is a follow-on to #19410 that further reduces the memory usage of
our reachability constraints. When finishing the building of a use-def
map, we walk through all of the "final" states and mark only those
reachability constraints as "used". We then throw away the interior TDD
nodes of any reachability constraints that weren't marked as used.

(This helps because we build up quite a few intermediate TDD nodes when
constructing complex reachability constraints. These nodes can never be
accessed if they were _only_ used as an intermediate TDD node. The
marking step ensures that we keep any nodes that ended up being referred
to in some accessible use-def map state.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dcreager dcreager dcreager approved these changes

@AlexWaygood AlexWaygood AlexWaygood approved these changes

@carljm carljm Awaiting requested review from carljm carljm is a code owner

@sharkdp sharkdp Awaiting requested review from sharkdp sharkdp is a code owner

Assignees

No one assigned

Labels

ty Multi-file analysis & type inference

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@MichaReiser @dcreager @AlexWaygood