SPARK-1632. Remove unnecessary boxing in compares in ExternalAppendOnlyM... #559
Conversation
|
Merged build triggered. |
There was a problem hiding this comment.
Worth saving the hashCodes rather than recomputing? it might be non-trivial
|
Merged build started. |
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
|
LGTM |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
|
Thanks, merged. |
…lyM... ...ap Author: Sandy Ryza <[email protected]> Closes #559 from sryza/sandy-spark-1632 and squashes the following commits: a6cd352 [Sandy Ryza] Only compute hashes once 04e3884 [Sandy Ryza] SPARK-1632. Remove unnecessary boxing in compares in ExternalAppendOnlyMap (cherry picked from commit 87cf35c) Signed-off-by: Patrick Wendell <[email protected]>
There was a problem hiding this comment.
It is better to use Integer.compare instead of writing these manually.
There was a problem hiding this comment.
Java 7 only :(
http://docs.oracle.com/javase/7/docs/api/java/lang/Integer.html#compare(int, int)
Guava has Ints.compare() though:
http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/primitives/Ints.html#compare(int, int)
There was a problem hiding this comment.
We could also write our own Utils.compare, though this is only used in 2 places so I think it's fine to just write it out directly. Also this is a super hot code path so maybe it's better to just avoid function calls altogether (maybe, maybe not).
…lyM... ...ap Author: Sandy Ryza <[email protected]> Closes apache#559 from sryza/sandy-spark-1632 and squashes the following commits: a6cd352 [Sandy Ryza] Only compute hashes once 04e3884 [Sandy Ryza] SPARK-1632. Remove unnecessary boxing in compares in ExternalAppendOnlyMap
1. The encrypt dockerhub user/password is wrong. It should not contain \n. 2. The image build job pipline is changed to `periodic-10/22`. 3. Correct jinjia format.
Change the implementation of `createTable` to avoid escaping of special chars in `UnresolvedTableSpec.location`. This field should contain the original user-provided `path` option and not the URI that is constructed by the `buildStorageFormatFromOptions()` call.
In addition this commit extends `SparkFunSuite` and `SQLTestUtils` to allow creating temporary directories with a custom prefix. This can be used to create temporary directories with special chars.
Bug fix. The following code would result in the creation of a table that is stored in `/tmp/test%table` instead of `/tmp/test table`:
```
spark.catalog.createTable("testTable", source = "parquet", schema = new StructType().add("id", "int"), description = "", options = Map("path" -> "/tmp/test table"))
```
Note that this was not consistent with the SQL API, e.g. `create table testTable(id int) using parquet location '/tmp/test table'`
Yes. The previous behaviour would result in table path be escaped. After this change the path will not be escaped.
Updated existing test in `CatalogSuite`.
No
Closes apache#47976 from cstavr/location-double-escaping.
Authored-by: Christos Stavrakakis <[email protected]>
(cherry picked from commit dc3333b)
Signed-off-by: Wenchen Fan <[email protected]>
Co-authored-by: Christos Stavrakakis <[email protected]>
6 participants
...ap