Commit eb5dc74
[SPARK-28097][SQL] Map ByteType to SMALLINT for PostgresDialect
## What changes were proposed in this pull request?
PostgreSQL doesn't have `TINYINT`, which would map directly, but `SMALLINT`s are sufficient for uni-directional translation.
A side-effect of this fix is that `AggregatedDialect` is now usable with multiple dialects targeting `jdbc:postgresql`, as `PostgresDialect.getJDBCType` no longer throws (for which reason backporting this fix would be lovely):
https://github.com/apache/spark/blob/1217996f1574f758d8cccc1c4e3846452d24b35b/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala#L42
`dialects.flatMap` currently throws on the first attempt to get a JDBC type preventing subsequent dialects in the chain from providing an alternative.
## How was this patch tested?
Unit tests.
Closes #24845 from mojodna/postgres-byte-type-mapping.
Authored-by: Seth Fitzsimmons <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>1 parent 28774cd commit eb5dc74
3 files changed
Lines changed: 15 additions & 6 deletions
File tree
- external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc
- sql/core/src
- main/scala/org/apache/spark/sql/jdbc
- test/scala/org/apache/spark/sql/jdbc
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
206 | 206 | | |
207 | 207 | | |
208 | 208 | | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
209 | 222 | | |
Lines changed: 1 addition & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
73 | 73 | | |
74 | 74 | | |
75 | 75 | | |
76 | | - | |
| 76 | + | |
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
83 | | - | |
84 | 83 | | |
85 | 84 | | |
86 | 85 | | |
| |||
Lines changed: 1 addition & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
857 | 857 | | |
858 | 858 | | |
859 | 859 | | |
860 | | - | |
861 | | - | |
862 | | - | |
863 | | - | |
| 860 | + | |
864 | 861 | | |
865 | 862 | | |
866 | 863 | | |
| |||
0 commit comments