HiveSync mode GLUE fails with Flink

[ertanden]

Bug Description

What happened:

Failure in Glue sync, constructor not found.

The constructor in AwsGlueCatalogSyncTool needs 3 arguments, the last one being an optional of metaClient.

  public AwsGlueCatalogSyncTool(Properties props, Configuration hadoopConf, Option<HoodieTableMetaClient> metaClient) {
    super(props, hadoopConf, metaClient);
  }

But in HiveSyncContext when the AwsGlueCatalogSyncTool is constructed with reflection, the metaClient argument is not provided.

  public HiveSyncTool hiveSyncTool() {
    HiveSyncMode syncMode = HiveSyncMode.of(props.getProperty(HIVE_SYNC_MODE.key()));
    if (syncMode == HiveSyncMode.GLUE) {
      return ((HiveSyncTool) ReflectionUtils.loadClass(AWS_GLUE_CATALOG_SYNC_TOOL_CLASS,
          new Class<?>[] {Properties.class, org.apache.hadoop.conf.Configuration.class},
          props, hiveConf));
    }
    return new HiveSyncTool(props, hiveConf);
  }

What you expected:

Glue Sync works with Flink.

Steps to reproduce:

1. Deploy Hudi with Flink bundle as uber jar
2. Create a table with 'hive_sync.mode' = 'GLUE'

Environment

Hudi version: 1.1.1 (flink bundle in uber jar)
Query engine: Flink 1.20 on AWS Managed Flink
Relevant configs: 'hive_sync.mode' = 'GLUE'

Logs and Stack Trace

No response

[danny0405]

thanks for the feedback, do you think you can contribute a patch to fix this bug?

[KiteSoar]

Hi @danny0405, I can contribute a patch to fix this bug. It seems like a simple fix by adjusting the constructor parameters when instantiating the Glue catalog sync tool.

[danny0405]

@KiteSoar Thanks for the help, will help for the review.