Skip to content

Do not make storage_descriptor require all arguments for TableBuilder #67

@sananguliyev

Description

@sananguliyev

storage_descriptor should not require all arguments for table builder because of virtual view. e.g.

table = TableBuilder(
    table_name="test_view",
    db_name="default",
    owner="test",
    table_type="VIRTUAL_VIEW",
    storage_descriptor=storage_descriptor,
    view_expanded_text="select * from test",
    view_original_text="select * from test"
).build()

When the user wants to create a virtual view, he/she should be able to just pass columns to storage descriptor instead of everything.

Instead of

storage_descriptor = StorageDescriptorBuilder(
    columns=columns,
    location="s3a://path/to/file",
    input_format="org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat",
    output_format="org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat",
    serde_info=serde_info,
).build()

user should pass just

storage_descriptor = StorageDescriptorBuilder(
    columns=columns,
).build()

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions