Commit e613f08
[SPARK-37867][SQL][FOLLOWUP] Compile aggregate functions for build-in DB2 dialect
### What changes were proposed in this pull request?
This PR follows up #35166.
The previously referenced DB2 documentation is incorrect, resulting in the lack of compile that supports some aggregate functions.
The correct documentation is https://www.ibm.com/docs/en/db2/11.5?topic=af-regression-functions-regr-avgx-regr-avgy-regr-count
### Why are the changes needed?
Make build-in DB2 dialect support complete aggregate push-down more aggregate functions.
### Does this PR introduce _any_ user-facing change?
'Yes'.
Users could use complete aggregate push-down with build-in DB2 dialect.
### How was this patch tested?
New tests.
Closes #35520 from beliefer/SPARK-37867_followup.
Authored-by: Jiaan Geng <beliefer@163.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 15532c7 commit e613f08
11 files changed
Lines changed: 123 additions & 83 deletions
File tree
- external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/v2
- sql/core/src/main/scala/org/apache/spark/sql/jdbc
Lines changed: 9 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
100 | 109 | | |
Lines changed: 4 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
| 101 | + | |
101 | 102 | | |
| 103 | + | |
102 | 104 | | |
| 105 | + | |
103 | 106 | | |
| 107 | + | |
104 | 108 | | |
Lines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
| 94 | + | |
94 | 95 | | |
| 96 | + | |
95 | 97 | | |
| 98 | + | |
96 | 99 | | |
| 100 | + | |
97 | 101 | | |
| 102 | + | |
98 | 103 | | |
| 104 | + | |
99 | 105 | | |
| 106 | + | |
100 | 107 | | |
Lines changed: 35 additions & 28 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
386 | 386 | | |
387 | 387 | | |
388 | 388 | | |
389 | | - | |
390 | | - | |
391 | | - | |
392 | | - | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
393 | 394 | | |
394 | 395 | | |
395 | 396 | | |
| |||
401 | 402 | | |
402 | 403 | | |
403 | 404 | | |
404 | | - | |
405 | | - | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
406 | 408 | | |
407 | | - | |
408 | | - | |
| 409 | + | |
| 410 | + | |
409 | 411 | | |
410 | 412 | | |
411 | 413 | | |
| |||
417 | 419 | | |
418 | 420 | | |
419 | 421 | | |
420 | | - | |
421 | | - | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
422 | 425 | | |
423 | | - | |
424 | | - | |
| 426 | + | |
| 427 | + | |
425 | 428 | | |
426 | 429 | | |
427 | 430 | | |
| |||
433 | 436 | | |
434 | 437 | | |
435 | 438 | | |
436 | | - | |
437 | | - | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
438 | 442 | | |
439 | | - | |
440 | | - | |
| 443 | + | |
| 444 | + | |
441 | 445 | | |
442 | 446 | | |
443 | 447 | | |
| |||
449 | 453 | | |
450 | 454 | | |
451 | 455 | | |
452 | | - | |
453 | | - | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
454 | 459 | | |
455 | | - | |
456 | | - | |
| 460 | + | |
| 461 | + | |
457 | 462 | | |
458 | 463 | | |
459 | 464 | | |
| |||
465 | 470 | | |
466 | 471 | | |
467 | 472 | | |
468 | | - | |
469 | | - | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
470 | 476 | | |
471 | | - | |
472 | | - | |
| 477 | + | |
| 478 | + | |
473 | 479 | | |
474 | 480 | | |
475 | 481 | | |
| |||
481 | 487 | | |
482 | 488 | | |
483 | 489 | | |
484 | | - | |
485 | | - | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
486 | 493 | | |
487 | | - | |
488 | | - | |
| 494 | + | |
| 495 | + | |
489 | 496 | | |
490 | 497 | | |
491 | 498 | | |
| |||
Lines changed: 19 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
40 | 59 | | |
41 | 60 | | |
42 | 61 | | |
| |||
Lines changed: 10 additions & 13 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
36 | | - | |
| 37 | + | |
37 | 38 | | |
38 | | - | |
39 | | - | |
40 | | - | |
| 39 | + | |
| 40 | + | |
41 | 41 | | |
42 | | - | |
43 | | - | |
44 | | - | |
| 42 | + | |
| 43 | + | |
45 | 44 | | |
46 | | - | |
47 | | - | |
48 | | - | |
| 45 | + | |
| 46 | + | |
49 | 47 | | |
50 | | - | |
51 | | - | |
| 48 | + | |
52 | 49 | | |
53 | 50 | | |
54 | 51 | | |
| |||
72 | 69 | | |
73 | 70 | | |
74 | 71 | | |
75 | | - | |
| 72 | + | |
76 | 73 | | |
77 | 74 | | |
78 | 75 | | |
| |||
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
43 | 46 | | |
44 | 47 | | |
45 | 48 | | |
| |||
Lines changed: 9 additions & 12 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
44 | | - | |
| 45 | + | |
45 | 46 | | |
46 | | - | |
47 | | - | |
48 | | - | |
| 47 | + | |
| 48 | + | |
49 | 49 | | |
50 | | - | |
51 | | - | |
52 | | - | |
| 50 | + | |
| 51 | + | |
53 | 52 | | |
54 | | - | |
55 | | - | |
56 | | - | |
| 53 | + | |
| 54 | + | |
57 | 55 | | |
58 | | - | |
59 | | - | |
| 56 | + | |
60 | 57 | | |
61 | 58 | | |
62 | 59 | | |
| |||
Lines changed: 17 additions & 21 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
37 | 40 | | |
38 | 41 | | |
39 | 42 | | |
40 | | - | |
| 43 | + | |
41 | 44 | | |
42 | | - | |
43 | | - | |
44 | | - | |
| 45 | + | |
| 46 | + | |
45 | 47 | | |
46 | | - | |
47 | | - | |
48 | | - | |
| 48 | + | |
| 49 | + | |
49 | 50 | | |
50 | | - | |
51 | | - | |
52 | | - | |
| 51 | + | |
| 52 | + | |
53 | 53 | | |
54 | | - | |
55 | | - | |
56 | | - | |
| 54 | + | |
| 55 | + | |
57 | 56 | | |
58 | | - | |
59 | | - | |
60 | | - | |
| 57 | + | |
| 58 | + | |
61 | 59 | | |
62 | | - | |
63 | | - | |
64 | | - | |
| 60 | + | |
| 61 | + | |
65 | 62 | | |
66 | | - | |
67 | | - | |
| 63 | + | |
68 | 64 | | |
69 | 65 | | |
70 | 66 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
| 39 | + | |
39 | 40 | | |
40 | 41 | | |
41 | 42 | | |
| |||
0 commit comments