Commit eb79fc4
[SPARK-49204][SQL][FOLLOWUP] Fix IndexOutOfBoundsException when dealing with surrogate pairs
### What changes were proposed in this pull request?
Modified the `StringLocate` ICU codepath and added a codepoint count check for the provided offset.
### Why are the changes needed?
Currently, doing the following throws an `java.lang.IndexOutOfBoundsException`.
```sql
select position('asd' COLLATE UNICODE, 'a', 10)
```
while the correct behavior is to return an false match result (0).
### Does this PR introduce _any_ user-facing change?
Yes, fixes the `java.lang.IndexOutOfBoundsException` error.
### How was this patch tested?
New test cases in this PR.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #47871 from stevomitric/stevomitric/fix-position-out-of-bounds.
Authored-by: Stevo Mitric <stevo.mitric@databricks.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>1 parent e1b5d65 commit eb79fc4
2 files changed
Lines changed: 26 additions & 0 deletions
File tree
- common/unsafe/src
- main/java/org/apache/spark/sql/catalyst/util
- test/java/org/apache/spark/unsafe/types
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
671 | 671 | | |
672 | 672 | | |
673 | 673 | | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
674 | 679 | | |
675 | 680 | | |
676 | 681 | | |
| |||
Lines changed: 21 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2329 | 2329 | | |
2330 | 2330 | | |
2331 | 2331 | | |
| 2332 | + | |
| 2333 | + | |
| 2334 | + | |
| 2335 | + | |
| 2336 | + | |
| 2337 | + | |
| 2338 | + | |
| 2339 | + | |
| 2340 | + | |
| 2341 | + | |
| 2342 | + | |
| 2343 | + | |
| 2344 | + | |
| 2345 | + | |
| 2346 | + | |
| 2347 | + | |
| 2348 | + | |
| 2349 | + | |
| 2350 | + | |
| 2351 | + | |
| 2352 | + | |
2332 | 2353 | | |
2333 | 2354 | | |
2334 | 2355 | | |
| |||
0 commit comments