[TEST] Add avg_to_sum optimizer #19624

Dandandan · 2026-01-03T14:13:50Z

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Dandandan · 2026-01-03T14:13:58Z

run benchmarks

Dandandan · 2026-01-03T14:14:08Z

run benchmark tpch tpcds

alamb-ghbot · 2026-01-03T14:14:08Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing test_avg_to_sum (3696ba5) to 70daf88 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

Dandandan · 2026-01-03T14:21:22Z

run benchmark tpch

alamb-ghbot · 2026-01-03T14:51:28Z

🤖: Benchmark completed

Details

Comparing HEAD and test_avg_to_sum
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ test_avg_to_sum ┃    Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 0     │  2396.82 ms │      2382.92 ms │ no change │
│ QQuery 1     │   957.55 ms │       936.21 ms │ no change │
│ QQuery 2     │  1949.44 ms │      1885.29 ms │ no change │
│ QQuery 3     │  1168.00 ms │      1190.28 ms │ no change │
│ QQuery 4     │  2247.23 ms │      2331.69 ms │ no change │
│ QQuery 5     │ 27862.49 ms │     28126.82 ms │ no change │
│ QQuery 6     │  3991.74 ms │      3954.72 ms │ no change │
│ QQuery 7     │  3482.88 ms │      3469.35 ms │ no change │
└──────────────┴─────────────┴─────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary              ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 44056.14ms │
│ Total Time (test_avg_to_sum)   │ 44277.27ms │
│ Average Time (HEAD)            │  5507.02ms │
│ Average Time (test_avg_to_sum) │  5534.66ms │
│ Queries Faster                 │          0 │
│ Queries Slower                 │          0 │
│ Queries with No Change         │          8 │
│ Queries with Failure           │          0 │
└────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ test_avg_to_sum ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     1.47 ms │         1.47 ms │     no change │
│ QQuery 1     │    49.60 ms │        49.84 ms │     no change │
│ QQuery 2     │   134.14 ms │       130.44 ms │     no change │
│ QQuery 3     │   150.48 ms │       153.17 ms │     no change │
│ QQuery 4     │  1077.73 ms │      1105.02 ms │     no change │
│ QQuery 5     │  1387.28 ms │      1399.12 ms │     no change │
│ QQuery 6     │     1.45 ms │         1.42 ms │     no change │
│ QQuery 7     │    54.52 ms │        54.75 ms │     no change │
│ QQuery 8     │  1451.98 ms │      1482.76 ms │     no change │
│ QQuery 9     │  1883.39 ms │      1879.64 ms │     no change │
│ QQuery 10    │   342.77 ms │       347.30 ms │     no change │
│ QQuery 11    │   401.23 ms │       396.91 ms │     no change │
│ QQuery 12    │  1303.72 ms │      1339.79 ms │     no change │
│ QQuery 13    │  1922.34 ms │      1971.41 ms │     no change │
│ QQuery 14    │  1261.64 ms │      1298.36 ms │     no change │
│ QQuery 15    │  1254.67 ms │      1280.89 ms │     no change │
│ QQuery 16    │  2565.00 ms │      2559.65 ms │     no change │
│ QQuery 17    │  2539.23 ms │      2542.68 ms │     no change │
│ QQuery 18    │  5167.05 ms │      5121.51 ms │     no change │
│ QQuery 19    │   119.44 ms │       120.60 ms │     no change │
│ QQuery 20    │  1874.83 ms │      1860.55 ms │     no change │
│ QQuery 21    │  2205.30 ms │      2160.30 ms │     no change │
│ QQuery 22    │  3757.50 ms │      3735.69 ms │     no change │
│ QQuery 23    │ 19255.07 ms │     12102.45 ms │ +1.59x faster │
│ QQuery 24    │   208.89 ms │       208.00 ms │     no change │
│ QQuery 25    │   464.55 ms │       464.48 ms │     no change │
│ QQuery 26    │   222.85 ms │       210.55 ms │ +1.06x faster │
│ QQuery 27    │  2783.71 ms │      2684.54 ms │     no change │
│ QQuery 28    │ 23479.91 ms │     21687.18 ms │ +1.08x faster │
│ QQuery 29    │   978.08 ms │       972.18 ms │     no change │
│ QQuery 30    │  1353.48 ms │      1316.70 ms │     no change │
│ QQuery 31    │  1356.71 ms │      1335.55 ms │     no change │
│ QQuery 32    │  5333.66 ms │      5249.12 ms │     no change │
│ QQuery 33    │  5383.08 ms │      5374.95 ms │     no change │
│ QQuery 34    │  5729.93 ms │      5855.41 ms │     no change │
│ QQuery 35    │  1962.45 ms │      1925.77 ms │     no change │
│ QQuery 36    │    68.16 ms │        68.11 ms │     no change │
│ QQuery 37    │    44.18 ms │        44.70 ms │     no change │
│ QQuery 38    │    64.41 ms │        68.72 ms │  1.07x slower │
│ QQuery 39    │   102.16 ms │       107.16 ms │     no change │
│ QQuery 40    │    26.82 ms │        27.35 ms │     no change │
│ QQuery 41    │    22.67 ms │        23.49 ms │     no change │
│ QQuery 42    │    19.32 ms │        18.95 ms │     no change │
└──────────────┴─────────────┴─────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary              ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 99766.87ms │
│ Total Time (test_avg_to_sum)   │ 90738.64ms │
│ Average Time (HEAD)            │  2320.16ms │
│ Average Time (test_avg_to_sum) │  2110.20ms │
│ Queries Faster                 │          3 │
│ Queries Slower                 │          1 │
│ Queries with No Change         │         39 │
│ Queries with Failure           │          0 │
└────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ test_avg_to_sum ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 120.29 ms │       107.67 ms │ +1.12x faster │
│ QQuery 2     │  29.19 ms │        28.90 ms │     no change │
│ QQuery 3     │  37.79 ms │        33.62 ms │ +1.12x faster │
│ QQuery 4     │  28.35 ms │        29.06 ms │     no change │
│ QQuery 5     │  87.23 ms │        85.77 ms │     no change │
│ QQuery 6     │  19.76 ms │        19.86 ms │     no change │
│ QQuery 7     │ 229.88 ms │       222.71 ms │     no change │
│ QQuery 8     │  33.61 ms │        33.90 ms │     no change │
│ QQuery 9     │  96.25 ms │       102.38 ms │  1.06x slower │
│ QQuery 10    │  62.22 ms │        62.83 ms │     no change │
│ QQuery 11    │  18.08 ms │        17.36 ms │     no change │
│ QQuery 12    │  49.25 ms │        49.38 ms │     no change │
│ QQuery 13    │  46.55 ms │        45.82 ms │     no change │
│ QQuery 14    │  13.11 ms │        13.50 ms │     no change │
│ QQuery 15    │  23.76 ms │        24.08 ms │     no change │
│ QQuery 16    │  24.25 ms │        23.68 ms │     no change │
│ QQuery 17    │ 146.86 ms │       150.72 ms │     no change │
│ QQuery 18    │ 281.78 ms │       278.69 ms │     no change │
│ QQuery 19    │  37.04 ms │        37.71 ms │     no change │
│ QQuery 20    │  49.04 ms │        49.72 ms │     no change │
│ QQuery 21    │ 293.94 ms │       308.45 ms │     no change │
│ QQuery 22    │  17.15 ms │        18.45 ms │  1.08x slower │
└──────────────┴───────────┴─────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary              ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 1745.38ms │
│ Total Time (test_avg_to_sum)   │ 1744.26ms │
│ Average Time (HEAD)            │   79.34ms │
│ Average Time (test_avg_to_sum) │   79.28ms │
│ Queries Faster                 │         2 │
│ Queries Slower                 │         2 │
│ Queries with No Change         │        18 │
│ Queries with Failure           │         0 │
└────────────────────────────────┴───────────┘

alamb-ghbot · 2026-01-03T14:51:32Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing test_avg_to_sum (3696ba5) to 70daf88 diff using: tpch
Results will be posted here when complete

alamb-ghbot · 2026-01-03T14:52:09Z

🤖: Benchmark completed

Details

Comparing HEAD and test_avg_to_sum
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ test_avg_to_sum ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1     │ 192.91 ms │       183.89 ms │    no change │
│ QQuery 2     │  94.57 ms │        93.33 ms │    no change │
│ QQuery 3     │ 123.11 ms │       123.00 ms │    no change │
│ QQuery 4     │  75.46 ms │        75.36 ms │    no change │
│ QQuery 5     │ 170.49 ms │       173.54 ms │    no change │
│ QQuery 6     │  64.01 ms │        64.88 ms │    no change │
│ QQuery 7     │ 215.47 ms │       218.07 ms │    no change │
│ QQuery 8     │ 160.51 ms │       160.17 ms │    no change │
│ QQuery 9     │ 222.51 ms │       218.60 ms │    no change │
│ QQuery 10    │ 186.58 ms │       182.58 ms │    no change │
│ QQuery 11    │  72.07 ms │        73.35 ms │    no change │
│ QQuery 12    │ 115.92 ms │       116.68 ms │    no change │
│ QQuery 13    │ 220.65 ms │       217.47 ms │    no change │
│ QQuery 14    │  90.15 ms │        91.13 ms │    no change │
│ QQuery 15    │ 122.34 ms │       120.85 ms │    no change │
│ QQuery 16    │  56.01 ms │        55.05 ms │    no change │
│ QQuery 17    │ 273.43 ms │       272.48 ms │    no change │
│ QQuery 18    │ 310.43 ms │       310.03 ms │    no change │
│ QQuery 19    │ 135.77 ms │       131.47 ms │    no change │
│ QQuery 20    │ 124.74 ms │       123.41 ms │    no change │
│ QQuery 21    │ 260.27 ms │       263.38 ms │    no change │
│ QQuery 22    │  40.75 ms │        43.10 ms │ 1.06x slower │
└──────────────┴───────────┴─────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary              ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 3328.14ms │
│ Total Time (test_avg_to_sum)   │ 3311.82ms │
│ Average Time (HEAD)            │  151.28ms │
│ Average Time (test_avg_to_sum) │  150.54ms │
│ Queries Faster                 │         0 │
│ Queries Slower                 │         1 │
│ Queries with No Change         │        21 │
│ Queries with Failure           │         0 │
└────────────────────────────────┴───────────┘

alamb-ghbot · 2026-01-03T14:52:11Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing test_avg_to_sum (3696ba5) to 70daf88 diff using: tpcds
Results will be posted here when complete

alamb-ghbot · 2026-01-03T14:52:12Z

Benchmark script failed with exit code 1.

Last 10 lines of output:

Click to expand

BRANCH_NAME: HEAD
DATA_DIR: /home/alamb/arrow-datafusion/benchmarks/data
RESULTS_DIR: /home/alamb/arrow-datafusion/benchmarks/results/HEAD
CARGO_COMMAND: cargo run --release
PREFER_HASH_JOIN: true
***************************

Please prepare TPC-DS data first by following instructions:
  ./bench.sh data tpcds

alamb-ghbot · 2026-01-03T15:07:14Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing test_avg_to_sum (3696ba5) to 70daf88 diff using: tpch
Results will be posted here when complete

alamb-ghbot · 2026-01-03T15:22:24Z

🤖: Benchmark completed

Details

Comparing HEAD and test_avg_to_sum
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ test_avg_to_sum ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1     │ 190.80 ms │       184.97 ms │    no change │
│ QQuery 2     │  93.20 ms │        91.35 ms │    no change │
│ QQuery 3     │ 124.00 ms │       123.54 ms │    no change │
│ QQuery 4     │  74.37 ms │        76.79 ms │    no change │
│ QQuery 5     │ 174.09 ms │       172.09 ms │    no change │
│ QQuery 6     │  62.61 ms │        67.23 ms │ 1.07x slower │
│ QQuery 7     │ 211.20 ms │       216.35 ms │    no change │
│ QQuery 8     │ 162.49 ms │       157.12 ms │    no change │
│ QQuery 9     │ 226.89 ms │       216.79 ms │    no change │
│ QQuery 10    │ 182.02 ms │       181.50 ms │    no change │
│ QQuery 11    │  70.96 ms │        76.78 ms │ 1.08x slower │
│ QQuery 12    │ 116.34 ms │       115.90 ms │    no change │
│ QQuery 13    │ 211.90 ms │       206.59 ms │    no change │
│ QQuery 14    │  87.30 ms │        89.84 ms │    no change │
│ QQuery 15    │ 118.75 ms │       116.27 ms │    no change │
│ QQuery 16    │  56.04 ms │        59.37 ms │ 1.06x slower │
│ QQuery 17    │ 268.92 ms │       273.48 ms │    no change │
│ QQuery 18    │ 312.23 ms │       310.08 ms │    no change │
│ QQuery 19    │ 133.01 ms │       132.55 ms │    no change │
│ QQuery 20    │ 124.05 ms │       125.76 ms │    no change │
│ QQuery 21    │ 258.64 ms │       265.25 ms │    no change │
│ QQuery 22    │  42.04 ms │        41.43 ms │    no change │
└──────────────┴───────────┴─────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary              ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 3301.85ms │
│ Total Time (test_avg_to_sum)   │ 3301.04ms │
│ Average Time (HEAD)            │  150.08ms │
│ Average Time (test_avg_to_sum) │  150.05ms │
│ Queries Faster                 │         0 │
│ Queries Slower                 │         3 │
│ Queries with No Change         │        19 │
│ Queries with Failure           │         0 │
└────────────────────────────────┴───────────┘

Dandandan · 2026-01-03T15:44:43Z

run benchmarks

alamb-ghbot · 2026-01-03T16:38:36Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing test_avg_to_sum (3696ba5) to 70daf88 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb-ghbot · 2026-01-03T17:15:55Z

🤖: Benchmark completed

Details

Comparing HEAD and test_avg_to_sum
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ test_avg_to_sum ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0     │  2519.69 ms │      2412.55 ms │    no change │
│ QQuery 1     │   916.97 ms │       980.23 ms │ 1.07x slower │
│ QQuery 2     │  1951.62 ms │      1911.89 ms │    no change │
│ QQuery 3     │  1208.49 ms │      1156.55 ms │    no change │
│ QQuery 4     │  2294.80 ms │      2274.42 ms │    no change │
│ QQuery 5     │ 28058.14 ms │     28368.75 ms │    no change │
│ QQuery 6     │  3999.43 ms │      3992.87 ms │    no change │
│ QQuery 7     │  3669.61 ms │      3604.10 ms │    no change │
└──────────────┴─────────────┴─────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary              ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 44618.75ms │
│ Total Time (test_avg_to_sum)   │ 44701.37ms │
│ Average Time (HEAD)            │  5577.34ms │
│ Average Time (test_avg_to_sum) │  5587.67ms │
│ Queries Faster                 │          0 │
│ Queries Slower                 │          1 │
│ Queries with No Change         │          7 │
│ Queries with Failure           │          0 │
└────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ test_avg_to_sum ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     1.39 ms │         1.45 ms │     no change │
│ QQuery 1     │    49.60 ms │        49.93 ms │     no change │
│ QQuery 2     │   133.99 ms │       128.69 ms │     no change │
│ QQuery 3     │   155.68 ms │       154.59 ms │     no change │
│ QQuery 4     │  1068.30 ms │      1092.47 ms │     no change │
│ QQuery 5     │  1402.88 ms │      1364.99 ms │     no change │
│ QQuery 6     │     1.46 ms │         1.46 ms │     no change │
│ QQuery 7     │    53.38 ms │        55.18 ms │     no change │
│ QQuery 8     │  1500.49 ms │      1457.09 ms │     no change │
│ QQuery 9     │  1896.39 ms │      1865.01 ms │     no change │
│ QQuery 10    │   343.77 ms │       343.80 ms │     no change │
│ QQuery 11    │   404.14 ms │       398.06 ms │     no change │
│ QQuery 12    │  1330.99 ms │      1273.88 ms │     no change │
│ QQuery 13    │  1967.60 ms │      1977.09 ms │     no change │
│ QQuery 14    │  1284.78 ms │      1265.41 ms │     no change │
│ QQuery 15    │  1312.07 ms │      1231.56 ms │ +1.07x faster │
│ QQuery 16    │  2610.86 ms │      2609.06 ms │     no change │
│ QQuery 17    │  2629.06 ms │      2543.04 ms │     no change │
│ QQuery 18    │  5626.62 ms │      4820.77 ms │ +1.17x faster │
│ QQuery 19    │   122.53 ms │       115.64 ms │ +1.06x faster │
│ QQuery 20    │  1957.63 ms │      1877.93 ms │     no change │
│ QQuery 21    │  2247.05 ms │      2174.48 ms │     no change │
│ QQuery 22    │  3838.51 ms │      3749.95 ms │     no change │
│ QQuery 23    │ 12506.38 ms │     12124.22 ms │     no change │
│ QQuery 24    │   215.62 ms │       201.94 ms │ +1.07x faster │
│ QQuery 25    │   462.27 ms │       461.11 ms │     no change │
│ QQuery 26    │   214.88 ms │       209.50 ms │     no change │
│ QQuery 27    │  2718.10 ms │      2706.70 ms │     no change │
│ QQuery 28    │ 23599.48 ms │     21676.13 ms │ +1.09x faster │
│ QQuery 29    │   969.10 ms │       967.00 ms │     no change │
│ QQuery 30    │  1392.97 ms │      1292.50 ms │ +1.08x faster │
│ QQuery 31    │  1395.35 ms │      1351.66 ms │     no change │
│ QQuery 32    │  4941.83 ms │      5093.36 ms │     no change │
│ QQuery 33    │  5794.20 ms │      5563.58 ms │     no change │
│ QQuery 34    │  5793.89 ms │      5587.57 ms │     no change │
│ QQuery 35    │  2057.96 ms │      1843.88 ms │ +1.12x faster │
│ QQuery 36    │    66.62 ms │        65.53 ms │     no change │
│ QQuery 37    │    44.02 ms │        43.12 ms │     no change │
│ QQuery 38    │    67.86 ms │        66.98 ms │     no change │
│ QQuery 39    │   102.46 ms │       101.01 ms │     no change │
│ QQuery 40    │    26.89 ms │        25.31 ms │ +1.06x faster │
│ QQuery 41    │    22.56 ms │        22.52 ms │     no change │
│ QQuery 42    │    19.36 ms │        18.83 ms │     no change │
└──────────────┴─────────────┴─────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary              ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 94350.99ms │
│ Total Time (test_avg_to_sum)   │ 89973.97ms │
│ Average Time (HEAD)            │  2194.21ms │
│ Average Time (test_avg_to_sum) │  2092.42ms │
│ Queries Faster                 │          8 │
│ Queries Slower                 │          0 │
│ Queries with No Change         │         35 │
│ Queries with Failure           │          0 │
└────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ test_avg_to_sum ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 119.01 ms │       109.11 ms │ +1.09x faster │
│ QQuery 2     │  28.19 ms │        28.88 ms │     no change │
│ QQuery 3     │  32.43 ms │        31.84 ms │     no change │
│ QQuery 4     │  29.58 ms │        28.33 ms │     no change │
│ QQuery 5     │  85.83 ms │        85.51 ms │     no change │
│ QQuery 6     │  19.85 ms │        19.72 ms │     no change │
│ QQuery 7     │ 224.96 ms │       220.86 ms │     no change │
│ QQuery 8     │  33.00 ms │        32.15 ms │     no change │
│ QQuery 9     │ 101.18 ms │       100.04 ms │     no change │
│ QQuery 10    │  62.85 ms │        63.16 ms │     no change │
│ QQuery 11    │  17.68 ms │        16.47 ms │ +1.07x faster │
│ QQuery 12    │  51.41 ms │        48.49 ms │ +1.06x faster │
│ QQuery 13    │  46.54 ms │        45.99 ms │     no change │
│ QQuery 14    │  13.65 ms │        13.36 ms │     no change │
│ QQuery 15    │  24.19 ms │        24.11 ms │     no change │
│ QQuery 16    │  24.37 ms │        24.18 ms │     no change │
│ QQuery 17    │ 150.12 ms │       157.45 ms │     no change │
│ QQuery 18    │ 278.54 ms │       270.89 ms │     no change │
│ QQuery 19    │  36.52 ms │        36.76 ms │     no change │
│ QQuery 20    │  50.05 ms │        48.32 ms │     no change │
│ QQuery 21    │ 292.24 ms │       292.16 ms │     no change │
│ QQuery 22    │  17.54 ms │        17.71 ms │     no change │
└──────────────┴───────────┴─────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary              ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 1739.74ms │
│ Total Time (test_avg_to_sum)   │ 1715.51ms │
│ Average Time (HEAD)            │   79.08ms │
│ Average Time (test_avg_to_sum) │   77.98ms │
│ Queries Faster                 │         3 │
│ Queries Slower                 │         0 │
│ Queries with No Change         │        19 │
│ Queries with Failure           │         0 │
└────────────────────────────────┴───────────┘

Dandandan · 2026-01-03T17:44:19Z

run benchmarks

alamb-ghbot · 2026-01-03T18:20:46Z

🤖 ./gh_compare_branch.sh gh_compare_branch.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing test_avg_to_sum (3696ba5) to 70daf88 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb-ghbot · 2026-01-03T18:58:07Z

🤖: Benchmark completed

Details

Comparing HEAD and test_avg_to_sum
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ test_avg_to_sum ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0     │  2393.79 ms │      2375.08 ms │    no change │
│ QQuery 1     │   927.77 ms │       975.28 ms │ 1.05x slower │
│ QQuery 2     │  1856.33 ms │      1926.61 ms │    no change │
│ QQuery 3     │  1185.76 ms │      1170.30 ms │    no change │
│ QQuery 4     │  2272.15 ms │      2264.71 ms │    no change │
│ QQuery 5     │ 28076.49 ms │     28362.22 ms │    no change │
│ QQuery 6     │  4000.66 ms │      3980.85 ms │    no change │
│ QQuery 7     │  3502.05 ms │      3675.50 ms │    no change │
└──────────────┴─────────────┴─────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary              ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 44215.00ms │
│ Total Time (test_avg_to_sum)   │ 44730.57ms │
│ Average Time (HEAD)            │  5526.88ms │
│ Average Time (test_avg_to_sum) │  5591.32ms │
│ Queries Faster                 │          0 │
│ Queries Slower                 │          1 │
│ Queries with No Change         │          7 │
│ Queries with Failure           │          0 │
└────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ test_avg_to_sum ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     1.45 ms │         1.43 ms │     no change │
│ QQuery 1     │    50.82 ms │        49.37 ms │     no change │
│ QQuery 2     │   132.49 ms │       131.44 ms │     no change │
│ QQuery 3     │   155.53 ms │       155.47 ms │     no change │
│ QQuery 4     │  1052.83 ms │      1079.23 ms │     no change │
│ QQuery 5     │  1364.49 ms │      1347.68 ms │     no change │
│ QQuery 6     │     1.47 ms │         1.44 ms │     no change │
│ QQuery 7     │    54.74 ms │        54.72 ms │     no change │
│ QQuery 8     │  1442.53 ms │      1439.57 ms │     no change │
│ QQuery 9     │  1786.21 ms │      1840.30 ms │     no change │
│ QQuery 10    │   345.70 ms │       345.13 ms │     no change │
│ QQuery 11    │   397.40 ms │       396.70 ms │     no change │
│ QQuery 12    │  1275.23 ms │      1267.78 ms │     no change │
│ QQuery 13    │  1924.13 ms │      1964.17 ms │     no change │
│ QQuery 14    │  1243.60 ms │      1243.25 ms │     no change │
│ QQuery 15    │  1246.89 ms │      1215.77 ms │     no change │
│ QQuery 16    │  2523.07 ms │      2578.77 ms │     no change │
│ QQuery 17    │  2505.53 ms │      2515.34 ms │     no change │
│ QQuery 18    │  5515.45 ms │      4797.34 ms │ +1.15x faster │
│ QQuery 19    │   118.43 ms │       118.98 ms │     no change │
│ QQuery 20    │  1905.50 ms │      1870.71 ms │     no change │
│ QQuery 21    │  2204.56 ms │      2177.89 ms │     no change │
│ QQuery 22    │  3812.41 ms │      3709.26 ms │     no change │
│ QQuery 23    │ 15961.75 ms │     12062.68 ms │ +1.32x faster │
│ QQuery 24    │   212.08 ms │       202.98 ms │     no change │
│ QQuery 25    │   461.60 ms │       461.83 ms │     no change │
│ QQuery 26    │   205.52 ms │       215.76 ms │     no change │
│ QQuery 27    │  2788.81 ms │      2704.94 ms │     no change │
│ QQuery 28    │ 23761.36 ms │     21854.84 ms │ +1.09x faster │
│ QQuery 29    │   972.41 ms │       950.40 ms │     no change │
│ QQuery 30    │  1317.71 ms │      1270.59 ms │     no change │
│ QQuery 31    │  1334.74 ms │      1355.38 ms │     no change │
│ QQuery 32    │  4913.96 ms │      4907.62 ms │     no change │
│ QQuery 33    │  5489.48 ms │      5416.82 ms │     no change │
│ QQuery 34    │  5708.56 ms │      5573.93 ms │     no change │
│ QQuery 35    │  1922.19 ms │      1845.27 ms │     no change │
│ QQuery 36    │    64.89 ms │        65.14 ms │     no change │
│ QQuery 37    │    44.02 ms │        45.44 ms │     no change │
│ QQuery 38    │    64.79 ms │        64.79 ms │     no change │
│ QQuery 39    │   101.87 ms │       103.08 ms │     no change │
│ QQuery 40    │    25.22 ms │        25.65 ms │     no change │
│ QQuery 41    │    21.42 ms │        21.96 ms │     no change │
│ QQuery 42    │    19.09 ms │        18.42 ms │     no change │
└──────────────┴─────────────┴─────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary              ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 96451.95ms │
│ Total Time (test_avg_to_sum)   │ 89469.27ms │
│ Average Time (HEAD)            │  2243.07ms │
│ Average Time (test_avg_to_sum) │  2080.68ms │
│ Queries Faster                 │          3 │
│ Queries Slower                 │          0 │
│ Queries with No Change         │         40 │
│ Queries with Failure           │          0 │
└────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ test_avg_to_sum ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 119.58 ms │       106.73 ms │ +1.12x faster │
│ QQuery 2     │  30.55 ms │        29.73 ms │     no change │
│ QQuery 3     │  32.23 ms │        32.68 ms │     no change │
│ QQuery 4     │  31.05 ms │        28.05 ms │ +1.11x faster │
│ QQuery 5     │  87.28 ms │        86.96 ms │     no change │
│ QQuery 6     │  19.61 ms │        22.15 ms │  1.13x slower │
│ QQuery 7     │ 226.75 ms │       226.60 ms │     no change │
│ QQuery 8     │  30.97 ms │        31.92 ms │     no change │
│ QQuery 9     │  99.51 ms │       103.63 ms │     no change │
│ QQuery 10    │  61.61 ms │        62.49 ms │     no change │
│ QQuery 11    │  17.21 ms │        16.89 ms │     no change │
│ QQuery 12    │  50.01 ms │        49.44 ms │     no change │
│ QQuery 13    │  45.26 ms │        47.10 ms │     no change │
│ QQuery 14    │  13.48 ms │        13.42 ms │     no change │
│ QQuery 15    │  24.33 ms │        24.15 ms │     no change │
│ QQuery 16    │  23.91 ms │        24.14 ms │     no change │
│ QQuery 17    │ 146.10 ms │       154.73 ms │  1.06x slower │
│ QQuery 18    │ 274.98 ms │       279.85 ms │     no change │
│ QQuery 19    │  37.50 ms │        37.14 ms │     no change │
│ QQuery 20    │  50.35 ms │        50.32 ms │     no change │
│ QQuery 21    │ 302.12 ms │       309.79 ms │     no change │
│ QQuery 22    │  17.74 ms │        17.81 ms │     no change │
└──────────────┴───────────┴─────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary              ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)              │ 1742.09ms │
│ Total Time (test_avg_to_sum)   │ 1755.73ms │
│ Average Time (HEAD)            │   79.19ms │
│ Average Time (test_avg_to_sum) │   79.81ms │
│ Queries Faster                 │         2 │
│ Queries Slower                 │         2 │
│ Queries with No Change         │        18 │
│ Queries with Failure           │         0 │
└────────────────────────────────┴───────────┘

Add avg_to_sum optimizer

3696ba5

github-actions bot added the optimizer Optimizer rules label Jan 3, 2026

Dandandan mentioned this pull request Jan 3, 2026

TPCH q1 with no predicates is 2x slower than duckdb #18411

Open

Dandandan mentioned this pull request Jan 4, 2026

Convert AVG(col) to SUM(x) / COUNT(*) #19637

Open

[TEST] Add avg_to_sum optimizer #19624

Are you sure you want to change the base?

[TEST] Add avg_to_sum optimizer #19624

Conversation

Dandandan commented Jan 3, 2026

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

Dandandan commented Jan 3, 2026

Uh oh!

Dandandan commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

Dandandan commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

Dandandan commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

Dandandan commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

alamb-ghbot commented Jan 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants