[HUDI-2509] OverwriteNonDefaultsWithLatestAvroPayload doesn`t work when upsert data with some null value column #3761
Conversation
[HUDI-2513] OverwriteNonDefaultsWithLatestAvroPayload doesn`t work when upsert data with some null value column
There was a problem hiding this comment.
@peanut-chenzhong Thanks for the fix! Can you add a functional test case to cover this scenario?
Also, some tests in the TestCOWDataSource failed. Please check if they are related to this diff.
xushiyan
changed the title
|
@peanut-chenzhong can you rebase master please? the Azure CI failure probably due to a failing master commit. |
…en upsert data with some null value column #3761
|
@xushiyan PR rebased and added UT for this scenario |
|
@hudi-bot run azure |
|
@peanut-chenzhong : I see even without the fix, the added test succeeds. Objects.equals(value, defaultValue) returns true even for null, null. Can you check if the fix is actually required. |
@nsivabalan modified the UT, this fix is necessary cause the default value will be changed to JsonProperty. |
codope
left a comment
codope
left a comment
There was a problem hiding this comment.
@peanut-chenzhong
One checkstyle violation. Can ou please fix it?
| assertEquals(payload1.combineAndGetUpdateValue(delRecord1, schema).get(), record2); | ||
| assertFalse(payload2.combineAndGetUpdateValue(record1, schema).isPresent()); | ||
| } | ||
| @Test |
There was a problem hiding this comment.
There is a checkstyle violation. Please add an empty line separator between methods.
6 participants
[HUDI-2509] OverwriteNonDefaultsWithLatestAvroPayload doesn`t work when upsert data with some null value column
Committer checklist
Has a corresponding JIRA in PR title & commit
Commit message is descriptive of the change
CI is green
Necessary doc changes done or have another open PR
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.