[HUDI-3514] Rebase Data Skipping flow to rely on MT Column Stats index#4948

Merged

yihua merged 23 commits intoapache:masterfrom

onehouseinc:ak/mtmod-idx-1

Mar 15, 2022

Commits on Mar 14, 2022

Added `columnName` to `ColumnStatsMetadata` payload
Alexey Kudinkin
committed
Populate `columnName` w/in `ColumnStats` payload
Alexey Kudinkin
committed
Scaffolded method to lookup candidate files based on the column statistics leveraging MT ColStats index
Alexey Kudinkin
committed
Fixing `TestLayoutOptimization` test enable MT and CS index
Alexey Kudinkin
committed
Added `HoodieDatasetUtils`;
Alexey Kudinkin
committed
Replaced all string literals with appropriate constants refs
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Killing dead-code
Alexey Kudinkin
committed
Fixing compilation
Alexey Kudinkin
committed
Cleaning up unused imports
Alexey Kudinkin
committed
Fixed `TablePathUtils` to correctly identify MT;
Alexey Kudinkin
committed
Transpose only columns referenced w/in filtering expressions;
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Tidying up (more)
Alexey Kudinkin
committed
Removing dead-code
Alexey Kudinkin
committed
Added test for Data Skipping flow w/in `HoodieFileIndex`
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Reverting recent `getTableAvroSchema` change
Alexey Kudinkin
committed
Fixing test record reader incorrectly augmenting schema with meta-fields
Alexey Kudinkin
committed

Commits on Mar 15, 2022

Fixed COW Hive InputFormat to leverage appropriate API of the `TableSchemaResolver`
Alexey Kudinkin
committed
Deprecated some APIs;
Alexey Kudinkin
committed
Unused imports
Alexey Kudinkin
committed
Fixed typo
Alexey Kudinkin
committed