Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-1824

Cannot create hive lineage when using query "insert into table X select A.Y, B.Z from A,B"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 0.7-incubating, 0.8-incubating, 0.7.1-incubating, 0.8.1, 1.0.0, trunk
    • 1.3.0
    • atlas-core

    Description

      Problem:

      I am using atlas to create hive lineage, I found that when using the hive query "insert into table A select B.name ,C.name from B, C", the lineage between A, B and C cannot be created.

      Steps to reproduce:
      1. log in to the hive shell
      2. create hive table a, b,c and insert into data into a,b and c
      3. using query "insert into table datalake_demo.d9 select a.id as key, a.id, a.aname ,b.bname,c.cname from datalake_demo.a,datalake_demo.b,datalake_demo.c where a.id = b.id and b.id = c.id" to create hive table d9

      Expected Result:
      hive_process is created , input of the process is a, b, c, output of the process is datalake_demo.d9, and the process could be shown on atlas UI

      Actual Result:
      hive_process is created, input of the process is a, b, b, d9, output of the process is null, and the process could not be shown on atlas UI, according to the attached pic "failed_lineage.png" and "failed_lineage_input_output.png"

      PS: We are working on fixing the issue!

      Attachments

        1. failed_lineage.png
          36 kB
          qinglin,xia
        2. failed_lineage_input_output.png
          67 kB
          qinglin,xia
        3. 0001-fix-ATLAS-1824.patch
          2 kB
          Xinzhi,Luo

        Activity

          People

            Xinzhi,Luo Xinzhi,Luo
            xiaqinglin qinglin,xia
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: