Skip to content

lateral view explode missing relationship #636

@Light-Towers

Description

@Light-Towers

Describe the bug

  • lateral view explode missing the relationship between table dw.scr and dw.dest.

To Reproduce
test.py

# -*- coding: utf-8 -*-
from sqllineage.runner import LineageRunner
def test_create_as():
    sql = """

INSERT OVERWRITE table dw.dest partition (dt)
select
    a.nameid,
    get_json_object(b.alteration_list,'$.alterDate') as alter_date,
    get_json_object(b.alteration_list,'$.alterItem') as alter_item,
    get_json_object(b.alteration_list,'$.preAlter') as alter_before,
    get_json_object(b.alteration_list,'$.postAlter') as alter_after,
    current_date() as dt
from (
     select t1.nameid,
            split(regexp_replace(regexp_replace(t1.alterationList, '[\\[\\]]', ''), '(},\\{)', '}#\\{'), '#') as dd
     from (select nameid, alterationlist
           from dw.src
           where alterationlist is not null
             and alterationlist not rlike '暂无数据'
             and alterationlist != '[]'
             and nameid is not null
          ) t1
 ) a lateral view explode(a.dd) b as alteration_list;

    """
    result = LineageRunner(sql, 'hive')

    result.print_column_lineage()
    print(result.source_tables)
    print(result.target_tables)

if __name__ == "__main__":
    test_create_as()
============================= test session starts =============================
collecting ... collected 1 item

testGetSqllineage.py::test_create_as PASSED                              [100%]dw.dest.alter_after <- <default>.b.alteration_list
dw.dest.alter_before <- <default>.b.alteration_list
dw.dest.alter_date <- <default>.b.alteration_list
dw.dest.alter_item <- <default>.b.alteration_list
dw.dest.nameid <- a.nameid <- t1.nameid <- dw.src.nameid
[Table: dw.src]
[Table: dw.dest]


============================== 1 passed in 1.75s ==============================

Expected behavior
maybe output should like dw.dest.alter_date <- b.alteration_list <- a.dd <- dw.src.alterationList

Python version (available via python --version)

  • 3.10.11

SQLLineage version (available via sqllineage --version):

  • 1.5.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions