-
Notifications
You must be signed in to change notification settings - Fork 72
Simplify single-partition sorting logic #262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify single-partition sorting logic #262
Conversation
| ) | ||
|
|
||
| df_result = c.sql( | ||
| "SELECT * FROM df ORDER BY a ASC NULLS FIRST, b DESC NULLS LAST, c DESC NULLS LAST" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The old logic that we were using to decide if we could do single-partition sorts (not any(sort_null_first[1:])) was incorrect, but we weren't catching that in this test because we weren't checking the full dataframe's sorting. This new test should fail with the old logic but pass with this PR.
|
This should be good to go in, but would currently depend on nightly cuDF; going to leave this as a draft until 21.12 is released |
Codecov Report
@@ Coverage Diff @@
## main #262 +/- ##
==========================================
+ Coverage 95.89% 95.99% +0.09%
==========================================
Files 64 64
Lines 2730 2794 +64
Branches 408 420 +12
==========================================
+ Hits 2618 2682 +64
+ Misses 72 71 -1
- Partials 40 41 +1
Continue to review full report at Codecov.
|
|
After some more thought here, I actually think it makes sense to merge in changes that depend on nightly cuDF - at the moment, GPU support is still experimental and I feel like it should be expected to require the latest changes on cuDF's end. Hopefully in the future we can relax this requirement and allow for stable versions (with performance being the only implication), but for now I think this should be good to merge as long as we also make sure to emphasize the need for cuDF nightlies in docs on GPU support. |
Agreed I think for cuDF/GPU support we can be quite liberal in our experimentation and communicate that to end users. |
With rapidsai/cudf#9455 merged, cuDF now has handling for lists of
ascendingbools andna_position, making it consistent with Pandas.