Skip to content

chore(python): Bring Pyright's type completeness to 99.7%, enforce minimum 99% type coverage in CI#22965

Closed
MarcoGorelli wants to merge 2 commits intopola-rs:mainfrom
MarcoGorelli:pyright-cov
Closed

chore(python): Bring Pyright's type completeness to 99.7%, enforce minimum 99% type coverage in CI#22965
MarcoGorelli wants to merge 2 commits intopola-rs:mainfrom
MarcoGorelli:pyright-cov

Conversation

@MarcoGorelli
Copy link
Copy Markdown
Collaborator

@MarcoGorelli MarcoGorelli commented May 27, 2025

Follow-up to #22962

The idea is to use pyright --verifytypes to enforce type completeness across the library

With this change, it brings type completeness to 99.7%. Without it, we get

           "message": "Type is missing type annotation and could be inferred differently by type checkers\n  Inferred type is \"DataFrame\"",

The remaining types reported as "partially unknown" are:

  • jemalloc_conf
  • override
  • CredentialProviderAzure
  • OptionType
  • DataTypeMappings
  • DataTypeGroup
  • CredentialProviderAzure
  • polars.lazyframe.opt_flags._ (?)

Unfortunately, pyright doesn't give us a way to enforce minimum type completeness microsoft/pyright#10458, they suggested to make a tool which wraps Pyright and parses the output json. That's what I've done in pyright-cov

Potential alternatives:

  • just vendor the pyright_cov.py file into the Polars repo. It's very simple, only ~50 lines long, nothing fancy
  • don't enforce minimum type completeness at all

I've been trying something similar in NumPy, and they've suggested BasedPyright - will take a look at that and then update numpy/numpy#29065 (comment)

@github-actions github-actions Bot added internal An internal refactor or improvement python Related to Python Polars labels May 27, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented May 27, 2025

Codecov Report

Attention: Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 80.32%. Comparing base (35f6e9f) to head (e9d2385).
Report is 62 commits behind head on main.

Files with missing lines Patch % Lines
.../polars/io/cloud/credential_provider/_providers.py 0.00% 1 Missing ⚠️
py-polars/polars/lazyframe/opt_flags.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #22965      +/-   ##
==========================================
- Coverage   80.33%   80.32%   -0.02%     
==========================================
  Files        1682     1682              
  Lines      223120   223114       -6     
  Branches     2804     2803       -1     
==========================================
- Hits       179251   179223      -28     
- Misses      43201    43223      +22     
  Partials      668      668              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal An internal refactor or improvement python Related to Python Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant