Skip to content

Conversation

@huberylee
Copy link
Contributor

@huberylee huberylee commented May 27, 2022

What is the purpose of the pull request

Support Create/Show/Drop Index for Spark SQL

-- Create index on specific column
CREATE INDEX indexName ON table (col) AS indexType

-- Show indexes from specific table
SHOW INDEXES (FROM | IN) table

-- Drop specific index
DROP INDEX indexName ON table 

Reference:

Hive sql index syntax:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/AlterIndex

MySQL index syntax:
https://dev.mysql.com/doc/refman/8.0/en/create-index.html
https://dev.mysql.com/doc/refman/8.0/en/show-index.html
https://dev.mysql.com/doc/refman/8.0/en/drop-index.html

Brief change log

  • Add CreateIndex/CreateIndexCommand/CreateIndexExec/ShowIndexes/ShowIndexesCommand/ShowIndexesExec/DropIndex/DropIndexCommand/DropIndexExec
  • Add HoodieSpark3PlanStrategy
  • Modify HoodieAnalaysis/*AstBuilder
  • Add TestSecondaryIndex

Verify this pull request

  • Added TestSecondaryIndex to verify the change.

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@XuQianJin-Stars
Copy link
Contributor

I think #5339 this syntax is better.

@melin
Copy link

melin commented May 28, 2022

spark create/drop index:
https://github.com/apache/spark/blob/branch-3.3/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4

@huberylee huberylee force-pushed the HUDI-4165-support-index-sql branch from 00e8a86 to 05da6b8 Compare May 28, 2022 02:02
@XuQianJin-Stars
Copy link
Contributor

spark create/drop index: https://github.com/apache/spark/blob/branch-3.3/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4

This index syntax is incomplete and we may also need to consider the compatibility of multiple versions of spark.

@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants