[HUDI-4294][Stacked on 4293] Introduce build action to actually perform index data generation

[huberylee]

Change Logs

Introducing a new action type called build to actually perform index data generation. This action contains two steps as clustering action does:

• Generate action plan to clarify which files and which indexes need to be built;
• Execute build index according action plan generated by step one;

Call procedure will be implemented as well to show or run build action.

Classes in package org.apache.hudi.secondary.index.lucene.hadoop were copied from package org.apache.solr.hdfs.store in Apache Solr project.

Impact

User can use call show_build(table=> '$table'[, path => $path], limit => $limit, show_involved_partition => [true/false]) to list build commits, usecall run_build(table => '$table'[, path => $path], predicate => '$predicate', show_involved_partition => [true/false]) to trigger new build action if conditions are satisfied.

Risk level: none | low | medium | high**

none

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[huberylee]

@hudi-bot run azure

[nsivabalan]

@huberylee : seems interesting idea. but do we have an RFC on this already ?

[huberylee]

@huberylee : seems interesting idea. but do we have an RFC on this already ?

Yes, this PR is part of the secondary index implementation. RFC is in another PR: #5370

[nsivabalan]

👍 cool

[yihua]

Hi @huberylee , similar here, are you still actively working on this PR and resolving conflicts?

[huberylee]

@hudi-bot run azure

[hudi-bot]

CI report:

• abc0856 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[codope]

There is an INDEX action for this purpose now.