-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Remove some unsafe and allocations when creating PrimitiveArrays from Vec and from_trusted_len_iter
#9299
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
from_trusted_len_iterunsafe and allocations when creating PrimitiveArrays from Vec and from_trusted_len_iter
| }; | ||
| PrimitiveArray::from(data) | ||
| let nulls = | ||
| Some(NullBuffer::new(BooleanBuffer::new(null, 0, len))).filter(|n| n.null_count() > 0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note the
- no more
unsafe - Use avoid
vec![buffer]allocation
|
run benchmark arrow_statistics |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
run benchmark builder |
|
🤖 |
|
🤖: Benchmark completed Details
|
|
There doesn't seem to be any measurable difference in the benchmarks |
Which issue does this PR close?
ArrayDatawith direct Array construction, when possible #9298Rationale for this change
While reviewing #9294 from @Dandandan I noticed some other places where we can avoid making ArrayData and thus save some allocations (and
unsafe)I don't expect this to make a huge performance difference, but every little allocation helps, and I think the change is justified simply from the perspective of avoiding some more
unsafeWhat changes are included in this PR?
Construct primitive arrays directly
Are these changes tested?
By existing CI
Are there any user-facing changes?