Skip to content

Conversation

@coderfender
Copy link
Contributor

@coderfender coderfender commented Nov 25, 2025

Which issue does this PR close?

Closed #531 (Adds ANSI support for decimal inputs)

Rationale for this change

  1. Changes to add ANSI and Try eval mode support to Decimal Sums
  2. The code already has support to identify overflow and the changes are essentially passing in eval mode and raising an error when there is an overflow. (is_null , is_empty flags). The only update here is to consolidate the state variables to 2 (optional i128 and an is_null flag). Unlike int sum, spark leverages both the sum and is_empty state vars in all modes

What changes are included in this PR?

  1. Changes to SumDecimal
  2. CometSum update to support ANSI mode
  3. Unit tests (and fix existing tests)

How are these changes tested?

  1. Unit test on scala side

@coderfender coderfender changed the title support_ansi_sum_decimal_input feat: Support ANSI Decimal Input SUM Nov 25, 2025
@coderfender
Copy link
Contributor Author

@andygrove , Please kickoff CI whenever get a chance . This PR should essentially finish impl ANSI (along with #2600 ) for all numeric types

@coderfender coderfender changed the title feat: Support ANSI Decimal Input SUM feat: Support ANSI mode for SUM Decimal types Nov 25, 2025
@coderfender coderfender changed the title feat: Support ANSI mode for SUM Decimal types feat: Support ANSI mode SUM (Decimal types) Nov 25, 2025
@codecov-commenter
Copy link

codecov-commenter commented Nov 25, 2025

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.17%. Comparing base (f09f8af) to head (ab437f5).
⚠️ Report is 730 commits behind head on main.

Files with missing lines Patch % Lines
...main/scala/org/apache/comet/serde/aggregates.scala 60.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2826      +/-   ##
============================================
+ Coverage     56.12%   59.17%   +3.05%     
- Complexity      976     1478     +502     
============================================
  Files           119      167      +48     
  Lines         11743    15188    +3445     
  Branches       2251     2522     +271     
============================================
+ Hits           6591     8988    +2397     
- Misses         4012     4916     +904     
- Partials       1140     1284     +144     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderfender
Copy link
Contributor Author

Fixed test failures by converting state variables into option and is_empty as per spark semantics . @andygrove , Please kickoff CI whenever you get a chance.

@coderfender
Copy link
Contributor Author

Fixed scala 2.13 formatting failures

Copy link
Member

@martin-g martin-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coderfender
Copy link
Contributor Author

Thank you for the approval @martin-g

Seq(true, false).foreach { ansiEnabled =>
withSQLConf(
SQLConf.ANSI_ENABLED.key -> ansiEnabled.toString,
CometConf.getExprAllowIncompatConfigKey(classOf[Sum]) -> "true") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these tests need to set CometConf.getExprAllowIncompatConfigKey(classOf[Sum]) -> "true"?

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @coderfender. One nit - the tests shouldn't be setting CometConf.getExprAllowIncompatConfigKey(classOf[Sum]) -> "true" so would be good to remove those lines.

@andygrove
Copy link
Member

Thanks @coderfender. One nit - the tests shouldn't be setting CometConf.getExprAllowIncompatConfigKey(classOf[Sum]) -> "true" so would be good to remove those lines.

This can be done in a separate PR

@andygrove
Copy link
Member

Thanks for the review @martin-g

@andygrove andygrove merged commit fd0ab64 into apache:main Dec 5, 2025
113 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ANSI support in SUM and AVG

4 participants