[ty] improve "Did you mean?" suggestion selection algorithm #21808
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…proved edit distance algorithm) Co-Authored-By: Alex Waygood <[email protected]>
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-12-05 12:09:17.684474584 +0000
+++ new-output.txt 2025-12-05 12:09:21.638475206 +0000
@@ -158,7 +158,7 @@
callables_protocol.py:169:7: error[invalid-assignment] Object of type `def cb8_bad1(x: int) -> Any` is not assignable to `Proto8`
callables_protocol.py:186:5: error[invalid-assignment] Object of type `Literal["str"]` is not assignable to attribute `other_attribute` of type `int`
callables_protocol.py:187:5: error[unresolved-attribute] Unresolved attribute `xxx` on type `Proto9[P@decorator1, R@decorator1]`.
-callables_protocol.py:197:7: error[unresolved-attribute] Object of type `Proto9[Unknown, Unknown]` has no attribute `other_attribute2`
+callables_protocol.py:197:7: error[unresolved-attribute] Object of type `Proto9[Unknown, Unknown]` has no attribute `other_attribute2`: Did you mean `other_attribute`?
callables_protocol.py:238:8: error[invalid-assignment] Object of type `def cb11_bad1(x: int, y: str, /) -> Any` is not assignable to `Proto11`
callables_protocol.py:260:8: error[invalid-assignment] Object of type `def cb12_bad1(*args: Any, *, kwarg0: Any) -> None` is not assignable to `Proto12`
callables_protocol.py:284:27: error[invalid-assignment] Object of type `def cb13_no_default(path: str) -> str` is not assignable to `Proto13_Default`
@@ -289,8 +289,8 @@
dataclasses_order.py:50:4: error[unsupported-operator] Operator `<` is not supported between objects of type `DC1` and `DC2`
dataclasses_postinit.py:28:7: error[unresolved-attribute] Object of type `DC1` has no attribute `x`
dataclasses_postinit.py:29:7: error[unresolved-attribute] Object of type `DC1` has no attribute `y`
-dataclasses_slots.py:66:1: error[unresolved-attribute] Class `DC6` has no attribute `__slots__`
-dataclasses_slots.py:69:1: error[unresolved-attribute] Object of type `DC6` has no attribute `__slots__`
+dataclasses_slots.py:66:1: error[unresolved-attribute] Class `DC6` has no attribute `__slots__`: Did you mean `__class__`?
+dataclasses_slots.py:69:1: error[unresolved-attribute] Object of type `DC6` has no attribute `__slots__`: Did you mean `__class__`?
dataclasses_transform_class.py:63:1: error[invalid-assignment] Property `id` defined in `Customer1` is read-only
dataclasses_transform_class.py:66:8: error[missing-argument] No arguments provided for required parameters `id`, `name`
dataclasses_transform_class.py:66:18: error[too-many-positional-arguments] Too many positional arguments: expected 0, got 2
@@ -971,7 +971,7 @@
typeddicts_extra_items.py:310:5: error[type-assertion-failure] Type `list[tuple[str, int | str]]` does not match asserted type `list[tuple[str, object]]`
typeddicts_extra_items.py:311:5: error[type-assertion-failure] Type `list[int | str]` does not match asserted type `list[object]`
typeddicts_extra_items.py:326:25: error[invalid-assignment] Object of type `IntDict` is not assignable to `dict[str, int]`
-typeddicts_extra_items.py:329:52: error[invalid-key] Unknown key "bar" for TypedDict `IntDictWithNum` - did you mean "num"?
+typeddicts_extra_items.py:329:52: error[invalid-key] Unknown key "bar" for TypedDict `IntDictWithNum`: Unknown key "bar"
typeddicts_extra_items.py:330:32: error[invalid-assignment] Object of type `IntDictWithNum` is not assignable to `dict[str, int]`
typeddicts_extra_items.py:337:1: error[unresolved-attribute] Object of type `IntDictWithNum` has no attribute `clear`
typeddicts_extra_items.py:339:1: error[type-assertion-failure] Type `tuple[str, int]` does not match asserted type `Unknown`
|
|
mtshiba
force-pushed
the
improve-edit-distance
branch
from
31f7a76 to
ed1f1e1
Compare
mtshiba
force-pushed
the
improve-edit-distance
branch
from
ed1f1e1 to
a1cc3db
Compare
AlexWaygood
added
ty
Member
|
You might be interested in looking at #18705 (which was reverted because it caused big performance regressions, but the performance regressions weren't because of the Levenshtein implementation — they were because the PR tried to list all attributes on a type, which is expensive). In that PR, we adapted CPython's Levenshtein implementation directly, because it's now existed in CPython for several Python minor releases, so we know that it gives very good suggestions for Python code specifically |
mtshiba
changed the title
Collaborator
Author
|
I decided to change the purpose of the PR to improve |
Collaborator
Author
Ah, I didn't notice that. |
Member
|
I suspect that the performance regression would be much less if we tried that PR again now, because we have to infer the types of all attributes on all first-party types anyway these days (to check for Liskov violations). So nearly all the expensive analysis from that PR would just be work that we'd have to do anyway at some point (and would be cached after the first computation). But we haven't finished our Liskov implementation yet; I was planning on experimenting with it again after we've finished our Liskov implementation. Anyway, I think it's very much worth it to improve our Levenshtein algorithm in the meantime! But I'd still be inclined to just pull out the Levenshtein implementation from that PR, since we added lots of tests for it, and since an algorithm that CPython uses for Python code is likely to be better at giving suggestions for Python code than an algorithm rustc uses for Rust code. |
Member
(one big, possibly expensive, thing that we don't do yet with our Liskov implementation, but that we need to add soon, is inferring the types of all first-party implicit instance attributes.) |
Collaborator
Author
OK, so I'm going to close this. |
CodSpeed Performance ReportMerging #21808 will degrade performances by 12.13%Comparing Summary
Benchmarks breakdown
Footnotes
|
Collaborator
Author
|
Hmm, this algorithm also causes performance degradation. Sorry for taking up your time! |
Member
|
No worries at all, thanks for trying it out! I still think it's probably worth it to try to improve our current Levenshtein algorithm. I'd welcome a PR that just pulls out the Levenshtein algorithm from #18705 and uses it to improve our current |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This is a revival of #21780. After seeing #21780, I thought that perhaps we could reduce the number of incorrect hits by improving the edit distance calculation and candidate filtering.
I improved
did_you_meanby referring to rustc's suggestion selection algorithm1.Test Plan
new snapshots
Footnotes
https://doc.rust-lang.org/beta/nightly-rustc/src/rustc_span/edit_distance.rs.html ↩