-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-29320][TESTS] Compare sql/core module in JDK8/11 (Part 1)
#25992
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #111650 has finished for PR 25992 at commit
|
|
Woah .. huge PR but +1 from a cursory look |
| date strings 2508 2524 16 4.0 250.8 0.7X | ||
| parse dates from Dataset[String] 21884 21902 19 0.5 2188.4 0.1X | ||
| from_csv(timestamp) 27188 27723 477 0.4 2718.8 0.1X | ||
| from_csv(date) 21137 21191 84 0.5 2113.7 0.1X |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where these result did go anywhere? We need to keep this empty file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, this seems to be truncated mistakenly for some reason.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh, yea.
|
After resolve the conflicts, I'll regenerate the result and reopen this. |
|
Oops. Rebasing and force-pushing seems to lose the track. |
|
#26003 is made. |
What changes were proposed in this pull request?
This PR regenerates the following
sql/corebenchmarks in JDK8/11 to compare the result.(These are less time-consuming half of the all
sql/corebenchmarks)AggregateBenchmark
BloomFilterBenchmark
BuiltInDataSourceWriteBenchmark
CSVBenchmark
ColumnarBatchBenchmark
CompressionSchemeBenchmark
DataSourceReadBenchmark
DatasetBenchmark
DateTimeBenchmark
ExternalAppendOnlyUnsafeRowArrayBenchmark
ExtractBenchmark
FilterPushdownBenchmark
HashedRelationMetricsBenchmark
InExpressionBenchmark
JoinBenchmark
MakeDateTimeBenchmark
MiscBenchmark
OrcNestedSchemaPruningBenchmark
OrcV2NestedSchemaPruningBenchmark
ParquetNestedSchemaPruningBenchmark
PrimitiveArrayBenchmark
RangeBenchmark
SortBenchmark
UDFBenchmark
UnsafeArrayDataBenchmark
WideSchemaBenchmark
WideTableBenchmark
Why are the changes needed?
According to the result, there are some difference between JDK8/JDK11. For example,
ByteBuffer APIandDirectByteBufferperformance is switched like the following.JDK8
JDK11
Does this PR introduce any user-facing change?
No.
How was this patch tested?
This is a test-only PR. We need to run benchmark.