Skip to content

fix: Fix lazy concat horizontal didn't raise on mismatching heights after projection pushdown#27506

Open
nameexhaustion wants to merge 9 commits intomainfrom
nxs/concat-horizontal-strict
Open

fix: Fix lazy concat horizontal didn't raise on mismatching heights after projection pushdown#27506
nameexhaustion wants to merge 9 commits intomainfrom
nxs/concat-horizontal-strict

Conversation

@nameexhaustion
Copy link
Copy Markdown
Collaborator

@nameexhaustion nameexhaustion commented May 5, 2026


Reverts the default behavior of strict horizontal concat w/ projection pushdown to before #25371. That is, we will always project at least 1 column from each input.

For users that would like to allow pruning of hconcat inputs in projection pushdown (thereby potentially discarding shape errors), an env var POLARS_PROJECTION_PUSHDOWN_PRUNE_STRICT_HCONCAT_INPUTS can be set.

@github-actions github-actions Bot added fix Bug fix python Related to Python Polars rust Related to Rust Polars labels May 5, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

The uncompressed lib size after this PR is 61.3522 MB.

@nameexhaustion nameexhaustion changed the title fix: Fix lazy concat horizontal didn't raise on mismatching heights fix: Fix lazy concat horizontal didn't raise on mismatching heights after projection pushdown May 5, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

The uncompressed lib size after this PR is 61.3522 MB.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

❌ Patch coverage is 98.73418% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.05%. Comparing base (1156024) to head (9499cac).
⚠️ Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
...lan/src/plans/optimizer/projection_pushdown/mod.rs 98.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #27506      +/-   ##
==========================================
- Coverage   81.14%   81.05%   -0.10%     
==========================================
  Files        1834     1836       +2     
  Lines      254891   254959      +68     
  Branches     3177     3179       +2     
==========================================
- Hits       206823   206647     -176     
- Misses      47246    47488     +242     
- Partials      822      824       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nameexhaustion nameexhaustion marked this pull request as ready for review May 5, 2026 13:38
@nameexhaustion nameexhaustion marked this pull request as draft May 6, 2026 05:14
let IR::ExtContext { schema, .. } = storage.get(key) else {
break 'patch_ext_context;
};
IR::ExtContext { schema, .. } => {
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extcontext branch is unchanged

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

The uncompressed lib size after this PR is 61.3398 MB.

@nameexhaustion nameexhaustion marked this pull request as ready for review May 6, 2026 08:42
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

The uncompressed lib size after this PR is 60.8103 MB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Bug fix python Related to Python Polars rust Related to Rust Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lazy horizontal concat with projection does not raise shape error for strict=True

1 participant