[HUDI-2809] Introduce a checksum mechanism for validating hoodie.properties

[codope]

What is the purpose of the pull request

To detect partial writes on HDFS, this PR adds a new property which gets appended at the end of hoodie.properties file while creating or modiying table config. The value of the property is CRC of <database_name>.<table_name>. The PR also changes TypedProperties to maintain order of insertion.

Brief change log

• Add a new table config.
• Support the new config in UpgradeDowngrade.
• Ensure insertion order in TypedProperties.

Verify this pull request

(Please pick either of the following options)

This pull request is a trivial rework / code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added integration tests for end-to-end.
• Added HoodieClientWriteTest to verify the change.
• Manually verified the change by running a job locally.

Committer checklist

• Has a corresponding JIRA in PR title & commit

• Commit message is descriptive of the change

• CI is green

• Necessary doc changes done or have another open PR

• For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

[nsivabalan]

can you help understand this scenario:

1. lets say we take a backup. and delete original table config. start writing new table config w/ updates. but crashed mid -way. Until we restart the hudi writer, primary table config is in corrupted state and backup property file is in good state. So, during this, what does reader do? do they get routed to backup or original?
2. If in above case, if crash happens mid-way, lets say out of 10 entries, only 5 got added, and if readers are using the primary copy to read table props, wouldn't they be reading partial table props?
   Probably this has nothing to do w/ the checksum patch I guess. but wanted to understand in general.

[codope]

can you help understand this scenario:

1. lets say we take a backup. and delete original table config. start writing new table config w/ updates. but crashed mid -way. Until we restart the hudi writer, primary table config is in corrupted state and backup property file is in good state. So, during this, what does reader do? do they get routed to backup or original?
2. If in above case, if crash happens mid-way, lets say out of 10 entries, only 5 got added, and if readers are using the primary copy to read table props, wouldn't they be reading partial table props?
   Probably this has nothing to do w/ the checksum patch I guess. but wanted to understand in general.

1. Get routed to backup props.
2. You mean while writing props from backup file, some crash happened even before checksum match? In that case, yes partial files would be written to hdfs but the writer will fail due to IOException. Next time, when it comes up then props are recovered from the backup.

[nsivabalan]

btw, we might need to add the checksum property as part of the upgrade.

[nsivabalan]

I would like for us to solve 2 partial write failure cases.

Case1:
you take backup.
delete original props file
update in memory props file.
write to original props file: mid way process crashed.

With this, how do we ensure readers get routed to backup file which is prestine and not the original file which is corrupt.

Case2:
when taking backup, let's say process crashed. but file was partially written.

Guess here we don't need to worry, bcoz, original file is still intact and readers will anyways read the original prop file.

Also, do we need to add checksum to prop file with an explicit upgrade step ?

[nsivabalan]

also, can you update the description of the patch (for eg, it does not cover upgrade info)

[nsivabalan]

1 nit. once addressed you can land. also there are some CI failures. do check it out.

[hudi-bot]

CI report:

• d2e24f8 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[codope]

CI was a build failure due to maven download timing out. It succeeded in the latest commit.