Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4206

Complex view creation with subquery hits Preconditions check when computing column lineage.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.5.0, Impala 2.4.0, Impala 2.6.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:
      None

      Description

      Conditions in CREATE VIEW that trigger this bug

      • View definition references another view
      • View definition has a WHERE-clause subquery
      • Column lineage is enabled (the impalad startup option "-lineage_event_log_dir" is set to a non-empty string)

      Repro:

      create table base (i int);
      create view nested_view as select i from base;
      
      create view err_view as select * from nested_view a
      where not exists (select 1 from base b where a.i = b.i);
      Query: create view err_view as select * from nested_view a where not exists (select 1 from base b where a.i = b.i)
      Query submitted at: 2016-09-28 10:38:49 (Coordinator: http://tp:25000)
      ERROR: AnalysisException: null
      CAUSED BY: IllegalStateException: null
      

      Stack trace from the impalad log:

      I0928 10:38:49.470296  8677 jni-util.cc:169] com.cloudera.impala.common.AnalysisException
      	at com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:382)
      	at com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:337)
      	at com.cloudera.impala.service.Frontend.analyzeStmt(Frontend.java:887)
      	at com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:916)
      	at com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:150)
      Caused by: java.lang.IllegalStateException
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
      	at com.cloudera.impala.analysis.ColumnLineageGraph.getSourceBaseCols(ColumnLineageGraph.java:462)
      	at com.cloudera.impala.analysis.ColumnLineageGraph.computeResultPredicateDependencies(ColumnLineageGraph.java:429)
      	at com.cloudera.impala.analysis.ColumnLineageGraph.computeLineageGraph(ColumnLineageGraph.java:362)
      	at com.cloudera.impala.analysis.CreateOrAlterViewStmtBase.computeLineageGraph(CreateOrAlterViewStmtBase.java:168)
      	at com.cloudera.impala.analysis.CreateViewStmt.analyze(CreateViewStmt.java:62)
      	at com.cloudera.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:362)
      
      

      This issue was very likely introduced by the patch for IMPALA-976.
      This issue was already fixed by IMPALA-3940, but we should add a regression test for this bug as well.

        Activity

        Hide
        alex.behm Alexander Behm added a comment -

        commit a5e84ac01491f8cca8a229c4348a23ae87ddca61
        Author: Alex Behm <alex.behm@cloudera.com>
        Date: Wed Sep 28 13:21:22 2016 -0700

        IMPALA-4206: Add column lineage regression test.

        The underlying issue was already fixed in IMPALA-3940.
        This patch adds a new regression test to cover the IMPALA-4206.

        Change-Id: I5b164000c7b0ce7e2f296d168d75a6860f5963d8
        Reviewed-on: http://gerrit.cloudera.org:8080/4556
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        alex.behm Alexander Behm added a comment - commit a5e84ac01491f8cca8a229c4348a23ae87ddca61 Author: Alex Behm <alex.behm@cloudera.com> Date: Wed Sep 28 13:21:22 2016 -0700 IMPALA-4206 : Add column lineage regression test. The underlying issue was already fixed in IMPALA-3940 . This patch adds a new regression test to cover the IMPALA-4206 . Change-Id: I5b164000c7b0ce7e2f296d168d75a6860f5963d8 Reviewed-on: http://gerrit.cloudera.org:8080/4556 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Internal Jenkins

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development