[HUDI-4465] Optimizing file-listing sequence of Metadata Table#6016

Merged

codope merged 27 commits intoapache:masterfrom

onehouseinc:ak/mtd-fl-lst-opt

Sep 9, 2022

Commits on Sep 2, 2022

Replaced string paht w/ Hadoop's `Path` to avoid costly conversions in the hot-path
Alexey Kudinkin
committed
Avoid fetching `FileSystem.get` since it's unused;
Alexey Kudinkin
committed
Cache `DateFormatter`/`TimestampFormatter` to avoid loading these t/h reflection in the hot-path
Alexey Kudinkin
committed
Avoid re-instantiating `HoodieTableMetadata` twice w/in `BaseHoodieTableFileIndex`
Alexey Kudinkin
committed
Fixing compilation
Alexey Kudinkin
committed
Avoid creating costly Hadoop's `Path` object wherever possible, replacing it w/
Alexey Kudinkin
committed
Fixing compilation
Alexey Kudinkin
committed
Avoid loading defaults in Hadoop conf when init-ing HFile reader;
Alexey Kudinkin
committed
Short-circuit fetching partition path for non-partitioned tables (simply return "");
Alexey Kudinkin
committed
Cleaned up `BaseTableMetadata`
Alexey Kudinkin
committed
Avoid looking up `FileSystem` for every partition when listing partitioned table, instead do it just once
Alexey Kudinkin
committed
`lint`
Alexey Kudinkin
committed
Revert accidental change
Alexey Kudinkin
committed
Reverting experimental changes
Alexey Kudinkin
committed
Fixing compilation (for Spark 2.4)
Alexey Kudinkin
committed
Make `CachingPath` non `Serializable`;
Alexey Kudinkin
committed
Fixed tests
Alexey Kudinkin
committed
Fixed invalid cast in `HoodieTableMetaClient`
Alexey Kudinkin
committed
Fixing tests w/ invalid configuration
Alexey Kudinkin
committed
Fixing more tests w/ invalid configuration
Alexey Kudinkin
committed
Fixed configs in IT to properly mirror configs for DS cases
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Carve out exception for MT to avoid treating it as non-partitioned one
Alexey Kudinkin
committed

Commits on Sep 3, 2022

Fixed `SimpleKeyGenerator` to assert that configured primary-key and
Alexey Kudinkin
committed
Fixed improperly configured test
Alexey Kudinkin
committed
Fixed invalid partition-path configs in `TestHoodieDeltaStreamer` suite
Alexey Kudinkin
committed

Commits on Sep 7, 2022

Rebased `TimeTravelQuery` test to use Simple KG instead
Alexey Kudinkin
committed