Improve vectorized operations of `GroupColumn`

[Rachelint]

Is your feature request related to a problem or challenge?

Some good points about improving vectorized operations of GroupColumn from @Dandandan and @jayzhan211 in #12996 :

#12996 (comment)
#12996 (comment)
#12996 (comment)
#12996 (comment)

This pr is for tracking such points for conintuing improving performace of multi group by.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

[Rachelint]

I tried the possible optimizations forvectorized_equal_to:
#12996 (comment)
#12996 (comment)

But unfortunately seems no obvious improvement in clickbench, tpch and so on... although some improvement can see in benchmark for the function in #13539

For take + eq, use eq in arrow kernel indeed can get performance improve, but the cost of take seems unacceptable...

For optimizations for special case of input array (like all rows in the array needed, no nulls although it is nullable), it can see some improvement in the benchmark for function, but see no improvement in e2e benchmark(clickbench, tpch). I guess they may be not the hot path...

I plan to not make further attempts recently...

cc @Dandandan @alamb @jayzhan211

[Rachelint]

I tried the possible optimizations forvectorized_equal_to: #12996 (comment) #12996 (comment)

But unfortunately seems no obvious improvement in clickbench, tpch and so on... although some improvement can see in benchmark for the function in #13539

For take + eq, use eq in arrow kernel indeed can get performance improve, but the cost of take seems unacceptable...

For optimizations for special case of input array (like all rows in the array needed, no nulls although it is nullable), it can see some improvement in the benchmark for function, but see no improvement in e2e benchmark(clickbench, tpch). I guess they may be not the hot path...

I plan to not make further attempts recently...

cc @Dandandan @alamb @jayzhan211

@LeslieKid maybe you can try it if some other possible optimizations not included above.

[Dandandan]

Thanks @Rachelint for summarizing, that's interesting. One big difference between take + eq in join versus grouped aggregates seems to be that the ValueBuilders already are type erased, so maybe there is nothing to gain (and using take and eq just creates an extra copy.

It seems for the join code, this could actually be used more (and joining on string columns might be not so fast at the moment).

[Rachelint]

Thanks @Rachelint for summarizing, that's interesting. One big difference between take + eq in join versus grouped aggregates seems to be that the ValueBuilders already are type erased, so maybe there is nothing to gain (and using take and eq just creates an extra copy.

It seems for the join code, this could actually be used more (and joining on string columns might be not so fast at the moment).

🤔 I think maybe we can indeed try some solutions without take in join? @LeslieKid seems trying it.

[Dandandan]

Yeah som experiments around that would be nice.

Also see this issue:
#12131

[Dandandan]

Oh I saw @LeslieKid already commented on that issue 👍

[LeslieKid]

🤔 I think maybe we can indeed try some solutions without take in join? @LeslieKid seems trying it.

Yes, I am working on comparing arrays with indices in a for-loop (similar to the implementation of vectorized_equal_to() in multi group-by) instead of using take in hash join. See #13607