[HUDI-4691] Cleaning up duplicated classes in Spark 3.3 module#6550

Merged

codope merged 38 commits into

apache:masterfrom

onehouseinc:ak/spk3-dedup

Sep 14, 2022

Commits on Sep 10, 2022

Moved `HoodieSpark3Analysis` and required classes along to "hudi-spark3-common"
Alexey Kudinkin
committed
Removed duplication from Spark 3.3 module
Alexey Kudinkin
committed
Unified Scala library deps to not over-specify version in every module;
Alexey Kudinkin
committed
Moved `HoodieIdentifier` to "hudi-spark3-common";
Alexey Kudinkin
committed
Missing licenses
Alexey Kudinkin
committed
Provisioned new "hudi-spark3.2plus-common" project (to accommodate for classes shared by Spark >= 3.2 modules)
Alexey Kudinkin
committed
Moved clases pertaining to Spark >= 3.2 under "hudi-spark3.2plus-common"
Alexey Kudinkin
committed
Fixed invalid module refs
Alexey Kudinkin
committed
Fixing module id
Alexey Kudinkin
committed
Removed unnecessary exclusions
Alexey Kudinkin
committed
Cleaned up unused `HoodieSPark3ResolveReferences` rule
Alexey Kudinkin
committed
Added `HoodieCatalogUtils` support into `SparkAdapter`
Alexey Kudinkin
committed
Fixed `HoodieCatalog` for Spark 3.3
Alexey Kudinkin
committed
Cleaned up duplicated `AlterTableCommand`;
Alexey Kudinkin
committed
De-duplicated Spar's default source impls for Spark 3.2 and 3.3 (as `Spark32PlusDefaultSource`)
Alexey Kudinkin
committed
Fixed `HoodieSpark3_1AvroDeserializer` to properly propagate legacy rebasing-mode for Avro;
Alexey Kudinkin
committed
De-duped `TimeTravelRelation`
Alexey Kudinkin
committed
De-duped Hudi's `ParquetFileFormat` extensions for Spark >= 3.2
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Missing licenses
Alexey Kudinkin
committed
Fixing invalid ref
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Fixed bundles to include new common module;
Alexey Kudinkin
committed
Fixed invalid ref on "hbase-server"
Alexey Kudinkin
committed
Fixing compilation for Spark 3.x
Alexey Kudinkin
committed
Cleaning up "spark2" profile
Alexey Kudinkin
committed
Removed "hudi-cli" dep from "hudi-spark-examples"
Alexey Kudinkin
committed
Exclude "hbase-client", "hbase-common" transitively pulled in from Hive modules
Alexey Kudinkin
committed
Added enforcer rules to make sure no transitive HBase deps are pulled in (clashing w/ the HBase version we're dependent on)
Alexey Kudinkin
committed
Excluding HBase deps pulled in by Hive in "hudi-spark-bundle"
Alexey Kudinkin
committed
Consolidated exclusions in the root POM
Alexey Kudinkin
committed
Restored "spark2" and "spark3" profiles (for BWC);
Alexey Kudinkin
committed
Fixed all Hive deps to exclude any transitive HBase deps
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Fixing typo
Alexey Kudinkin
committed
Tidying up
Alexey Kudinkin
committed
Fixed Hudi IT scripts to properly alias bundle jars (having the format "hudi-spark2.4-bundle_2.11-..."
Alexey Kudinkin
committed

Commits on Sep 14, 2022

Unravel globbing adn replaced w/ module-ref placeholders
Alexey Kudinkin
committed