Add comparison support for Union arrays

[friendlymatthew]

Which issue does this PR close?

• Closes Add comparison support for Union arrays in the cmp kernel #8837
• Related to Support Union data types for row format #8828

Rationale for this change

This PR implements comparison functionality for Union arrays. This implementation follows a simple ordering strategy where unions are first compared by their type identifier, and only when type identifiers match are the actual values within those types compared

This approach handles both sparse and dense union modes correctly by using offsets when present (dense unions) or direct indices (sparse unions) to locate the appropriate child array values

[friendlymatthew]

cc @paddyhoran @alamb

[alamb]

Thanks @friendlymatthew -- this looks good to me. The only thing I think it needs are some tests of the error cases -- namely

1. Compare an out of bounds index (expect panic)
2. Try to compare incompatible union types

I also left some other small suggestions but I don't think they are needed

[alamb]

This is weird to have to re-check the DataTypes.

What would you think about adding UnionArray::fields() and UnionArray::mode() methods to make the code easier to work with?

[friendlymatthew]

This should be super quick to review: #8884

Somewhat related but it feels a bit weird that the following works without any notice to the user:

#[test]
fn test_union_fields() {
    let ids = vec![0, 1, 2];
    let field = Field::new("a", DataType::Binary, true);

    // different length of ids and fields (we zip so we truncate the longer vec)
    let _out = UnionFields::new(ids.clone(), vec![field.clone()]);

    // duplicate fields associated with different type ids!
    let _out = UnionFields::new(ids, vec![field.clone(), field]);
}

I feel like we could benefit from a bit more validation? We could leave UnionFields::new but also have a UnionFields::try_new that checks the above 🤔

[alamb]

Yes, I think that sounds like a good idea to me

We can even deprecate UnionFields::new to help people migrate over

[friendlymatthew]

Here it is: #8891

[friendlymatthew]

Here is another minor convenience improvement: #8895

[alamb]

I recommend adding more details to this message to help when people hit it -- specifically, I recommend

1. a separate message for different modes (and include the modes in the error message)
2. Add the fields ({fields:?} style) to the message

[alamb]

rather than a hash map you could potentially just use a 128 valued Vec<> indexed by the typeids -- since typeid is i8 you know there can be at most 128 values that might be faster to lookup than hashing/hash table

[friendlymatthew]

Hm so this was my first thought/approach as well, but I decided to use a hashmap because it avoids superfluous memory usage for sparse sets

Plus, I don't think this is a very hot path, so any perf differences wouldn't be super meaningful

[alamb]

Thank you @friendlymatthew

[alamb]

Thanks @friendlymatthew