Fix #11472: TypeVarTuple escapes method with very nested recursive tuple aliases#11497
Open
rchiodo wants to merge 1 commit into
Open
Fix #11472: TypeVarTuple escapes method with very nested recursive tuple aliases#11497rchiodo wants to merge 1 commit into
rchiodo wants to merge 1 commit into
Conversation
The tuple type-var transformer bailed out early (returning the type unchanged) whenever a tuple's container depth exceeded a fixed limit of 10. This was meant to prevent the analyzer from hanging on tuple types that grow without bound during constraint solving. However, legitimate code can produce bounded-but-deeply-nested tuples via recursive type aliases, and bailing out there silently dropped already-solved TypeVar substitutions, causing a solved TypeVar (e.g. OO) to escape into the inferred type and produce a spurious assignment error. Container depth cannot distinguish a legitimately deep tuple from a runaway one: a single substitution of a bounded recursive alias can jump the depth past the limit just as runaway recursion does. The real protection against unbounded tuple growth is already provided by the apply() recursion-count limit and the pending-type-var-scope mechanism (which leaves same-scope recursive type vars unexpanded), so the depth guard is redundant and is removed. Fixes microsoft#11472 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: sympy (https://github.com/sympy/sympy)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:315:25 - error: "Set" is not iterable
+ "__iter__" method not defined (reportGeneralTypeIssues)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:315:25 - error: "ConditionSet" is not iterable
+ "__iter__" method not defined (reportGeneralTypeIssues)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2030:20 - error: "__getitem__" method not defined on type "Basic" (reportIndexIssue)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2042:20 - error: "__getitem__" method not defined on type "Basic" (reportIndexIssue)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2055:20 - error: "__getitem__" method not defined on type "Basic" (reportIndexIssue)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2076:16 - error: Argument of type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" cannot be assigned to parameter "obj" of type "Sized" in function "len"
+ Type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" is not assignable to type "Sized"
+ "Set" is incompatible with protocol "Sized"
+ "__len__" is not present (reportArgumentType)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2077:20 - error: "__getitem__" method not defined on type "Basic" (reportIndexIssue)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2270:16 - error: Argument of type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" cannot be assigned to parameter "obj" of type "Sized" in function "len"
+ Type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" is not assignable to type "Sized"
+ "Set" is incompatible with protocol "Sized"
+ "__len__" is not present (reportArgumentType)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2351:16 - error: Argument of type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" cannot be assigned to parameter "obj" of type "Sized" in function "len"
+ Type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" is not assignable to type "Sized"
+ "Set" is incompatible with protocol "Sized"
+ "__len__" is not present (reportArgumentType)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2364:16 - error: Argument of type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" cannot be assigned to parameter "obj" of type "Sized" in function "len"
+ Type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" is not assignable to type "Sized"
+ "Set" is incompatible with protocol "Sized"
+ "__len__" is not present (reportArgumentType)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2370:16 - error: Argument of type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" cannot be assigned to parameter "obj" of type "Sized" in function "len"
+ Type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" is not assignable to type "Sized"
+ "Set" is incompatible with protocol "Sized"
+ "__len__" is not present (reportArgumentType)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2538:16 - error: Argument of type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" cannot be assigned to parameter "obj" of type "Sized" in function "len"
+ Type "Unknown | FiniteSet | Set | Intersection | Union | Complement | Any | ConditionSet" is not assignable to type "Sized"
+ "Set" is incompatible with protocol "Sized"
+ "__len__" is not present (reportArgumentType)
+ .../projects/sympy/sympy/solvers/tests/test_solveset.py:2541:20 - error: Argument of type "Unknown | Basic | Any" cannot be assigned to parameter "obj" of type "Sized" in function "len"
+ Type "Unknown | Basic | Any" is not assignable to type "Sized"
+ "Basic" is incompatible with protocol "Sized"
+ "__len__" is not present (reportArgumentType)
+ .../projects/sympy/sympy/stats/crv_types.py:2544:9 - error: Method "_cdf" overrides class "SingleContinuousDistribution" in an incompatible manner
+ Return type mismatch: base method returns type "None", override returns type "ComplexInfinity | NaN | Rational | Unknown | Zero | Expr"
+ Type "ComplexInfinity | NaN | Rational | Unknown | Zero | Expr" is not assignable to type "None"
+ "Expr" is not assignable to "None" (reportIncompatibleMethodOverride)
+ .../projects/sympy/sympy/stats/crv_types.py:2723:9 - error: Method "_cdf" overrides class "SingleContinuousDistribution" in an incompatible manner
+ Return type mismatch: base method returns type "None", override returns type "Expr | NaN | ComplexInfinity | Rational | Unknown | One | NegativeOne | Zero | Integer"
+ Type "Expr | NaN | ComplexInfinity | Rational | Unknown | One | NegativeOne | Zero | Integer" is not assignable to type "None"
+ "Expr" is not assignable to "None" (reportIncompatibleMethodOverride)
- .../projects/sympy/sympy/stats/drv.py:269:22 - error: Argument of type "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None, None, None]" cannot be assigned to parameter "iterable" of type "Iterable[_SupportsSumNoDefaultT@sum]" in function "sum"
+ .../projects/sympy/sympy/stats/drv.py:269:22 - error: Argument of type "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None, None, None]" cannot be assigned to parameter "iterable" of type "Iterable[_SupportsSumNoDefaultT@sum]" in function "sum"
- "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None, None, None]" is not assignable to "Iterable[_SupportsSumNoDefaultT@sum]"
+ "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None, None, None]" is not assignable to "Iterable[_SupportsSumNoDefaultT@sum]"
- Type parameter "_T_co@Iterable" is covariant, but "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None" is not a subtype of "_SupportsSumNoDefaultT@sum"
+ Type parameter "_T_co@Iterable" is covariant, but "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None" is not a subtype of "_SupportsSumNoDefaultT@sum"
- Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
+ Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
- Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
+ Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
- .../projects/sympy/sympy/stats/drv_types.py:293:16 - error: Operator "*" not supported for types "Expr" and "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic"
+ .../projects/sympy/sympy/stats/drv_types.py:293:16 - error: Operator "*" not supported for types "Expr" and "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic"
+ .../projects/sympy/sympy/stats/frv_types.py:139:17 - error: Argument of type "NaN | ComplexInfinity | Rational | Unknown | Expr" cannot be assigned to parameter "value" of type "int" in function "__setitem__"
+ Type "NaN | ComplexInfinity | Rational | Unknown | Expr" is not assignable to type "int"
+ "Expr" is not assignable to "int" (reportArgumentType)
- .../projects/sympy/sympy/stats/rv.py:473:25 - error: Argument of type "Expr | Lambda | Zero | One | Integral | Unknown | Probability | tuple[Unknown, ...] | Sum | ZeroMatrix | NaN | Piecewise | Basic | NegativeOne | Integer | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | int" cannot be assigned to parameter "args" of type "Expr | complex" in function "__new__"
+ .../projects/sympy/sympy/stats/rv.py:473:25 - error: Argument of type "Expr | Lambda | Zero | One | Integral | Unknown | Probability | tuple[Unknown, ...] | Sum | ZeroMatrix | Add | NaN | Piecewise | Basic | NegativeOne | Integer | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | int" cannot be assigned to parameter "args" of type "Expr | complex" in function "__new__"
- Type "Expr | Lambda | Zero | One | Integral | Unknown | Probability | tuple[Unknown, ...] | Sum | ZeroMatrix | NaN | Piecewise | Basic | NegativeOne | Integer | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | int" is not assignable to type "Expr | complex"
+ Type "Expr | Lambda | Zero | One | Integral | Unknown | Probability | tuple[Unknown, ...] | Sum | ZeroMatrix | Add | NaN | Piecewise | Basic | NegativeOne | Integer | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | int" is not assignable to type "Expr | complex"
- .../projects/sympy/sympy/stats/stochastic_process_types.py:1839:25 - error: Argument of type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | Integral | Any | int" cannot be assigned to parameter "args" of type "Expr | complex" in function "__new__"
... (truncated 832 lines) ...
|
rchiodo
commented
Jun 18, 2026
| return classType; | ||
| } | ||
|
|
||
| if (classType.priv.tupleTypeArgs) { |
Collaborator
Author
There was a problem hiding this comment.
Removing this early-return fixes the escaping TypeVar (1→0 errors), but it also deletes a documented performance safeguard whose comment explicitly claimed the recursion-count limit is insufficient ("will effectively hang the analyzer"). Consider a narrower, safe-bailout variant: keep a depth bound but make the bailout safe (e.g. return Unknown or still solve top-level TypeVars) so nothing escapes at any depth while preserving the perf guard. Since this is vendored, upstream-owned Pyright core (keep diffs minimal, coordinate upstream), prefer the safe-bailout variant or coordinate the bound removal upstream against the original motivating repro, and confirm that repro still terminates without the guard.
rchiodo
commented
Jun 18, 2026
| const configOptions = new ConfigOptions(Uri.empty()); | ||
|
|
||
| configOptions.defaultPythonVersion = pythonVersion3_12; | ||
| const analysisResults = TestUtils.typeAnalyzeSampleFiles(['typeVarTuple31.py'], configOptions); |
Collaborator
Author
There was a problem hiding this comment.
validateResults(analysisResults, 0) asserts only "no errors" — it would still pass if OO were replaced by some other wrong-but-non-erroring type. For a TypeVar-escape bug, add a reveal_type (or hover) assertion pinning forloop's concrete inferred type so the fix's correctness is self-evident and a future divergent-but-non-error regression is caught.
Collaborator
Author
|
Fix looks correct and well-targeted. Two non-blocking suggestions: (1) the removed depth guard was a documented performance safeguard in vendored Pyright core — please confirm the original motivating workload still terminates and consider coordinating upstream; (2) strengthen the test with a reveal_type assertion to pin the inferred type rather than only asserting zero errors. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes a bug where a
TypeVarTuplecould "escape" a method's signature when the receiver's type arguments contained deeply nested recursive tuple type aliases. In these cases, an unsolved TypeVar (e.g.OOfromunpack_then) leaked into the inferred type of an assignment, producing a spurious type error.How you figured out what to do
The repro chained parser combinators (
then/unpack_then) on values whose types involved recursive tuple aliases (Whitespace,Implementations,BlockItem). Tracing the failure showed thatTypeVarTransformerbailed out early when transforming tuple types whose container depth exceeded a hard-coded limit (maxTupleTypeArgRecursionDepth = 10). When that early-return fired mid-transformation, the method's TypeVars were never substituted, so they escaped into the result.Implementation
Removed the
maxTupleTypeArgRecursionDepthguard intypeUtils.ts:maxTupleTypeArgRecursionDepthconstant.getContainerDepth(classType) > maxTupleTypeArgRecursionDepthearly-return inTypeVarTransformer's tuple handling, so deeply nested tuples are transformed fully instead of being returned unmodified.The depth bail-out was originally a guard against effectively-infinite nesting, but the existing recursion-count limit already protects against runaway types, and the early return caused incorrect TypeVar solving in legitimate code.
Testing
Added
typeVarTuple31.pysample reproducing the escaping TypeVar with nested recursive tuple aliases, and wired it up asTypeVarTuple31intypeEvaluator6.test.ts(Python 3.12). The sample asserts zero errors; the final chained assignment previously failed becauseOOescaped. Ran the type-evaluator test suite to confirm the new test passes and no existingTypeVarTupletests regressed.Addresses
Fixes https://github.com/microsoft/pylance-release/issues/11472
Generated by fix_all_my_issues pipeline