Commit 70c1be3
committed
[SPARK-58732][ML] Avoid MLlib vector conversion in Normalizer
### What changes were proposed in this pull request?
Update ML Normalizer to calculate the norm and normalize values with ML vectors directly. The normalization implementation follows the existing MLlib Normalizer transform method, including cloning dense and sparse value arrays and reusing sparse indices.
### Why are the changes needed?
ML Normalizer currently converts every input vector from ML to MLlib and converts the normalized result back to ML. Avoiding those conversions reduces allocation and transform overhead.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The existing NormalizerSuite covers dense, sparse, zero-vector, and parameterized normalization behavior.
build/sbt 'mllib/testOnly org.apache.spark.ml.feature.NormalizerSuite'
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
Closes #57955 from zhengruifeng/SPARK-58732-normalizer-native-vector.
Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>
(cherry picked from commit cddd9ae)
Signed-off-by: Ruifeng Zheng <ruifengz@foxmail.com>1 parent 99302f5 commit 70c1be3
1 file changed
Lines changed: 38 additions & 5 deletions
Lines changed: 38 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | | - | |
27 | | - | |
28 | 27 | | |
29 | 28 | | |
30 | 29 | | |
| |||
56 | 55 | | |
57 | 56 | | |
58 | 57 | | |
59 | | - | |
60 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
61 | 94 | | |
62 | 95 | | |
63 | 96 | | |
| |||
0 commit comments