MAPREDUCE-7524 Addressed indeterminism in TestCombineFileInputFormat #8134
+171
−220
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…by removing assertions based on the internal implementation of HDFS.
Description of PR
org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat.testSplitPlacementForCompressedFilesmakes a number of assertions that are not guaranteed by internal implementation.For example, the test makes assumptions on the order in which racks are processed. Splits in the HDFS can appear in a different order depending on the internal state / implementation / metadata which is not guaranteed by the interface.
For example, there are assertions such as split.equals(splits.get(0)), which assumes a specific ordering of InputSplit objects that may vary depending on internal implementation.
This can be addressed by eliminating assertions based on the internal implementation / ways that the racks are split.
How was this patch tested?
existing tests that pass continue to pass and tests that fail continue to fail. The tests were inspected closely to ensure that the original assertions were followed while maintaining deterministic ordering
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?