Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-787

FilterPushDownRule::visitSubQuery does not consider aliased columns.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0, 0.9.0
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      I found a bug for FilterPushDownRule as follows:

      • Schema
        default> \d table1;
        
        table name: default.table1
        table path: hdfs://localhost:9010/tajo/warehouse/table1
        store type: CSV
        number of rows: 0
        volume: 60 B
        Options: 
        	'csvfile.delimiter'='|'
        
        schema: 
        id	INT4
        name	TEXT
        score	FLOAT4
        type	TEXT
        
        
        default> \d table2;
        
        table name: default.table2
        table path: hdfs://localhost:9010/tajo/warehouse/table2
        store type: CSV
        number of rows: 0
        volume: 61 B
        Options: 
        	'csvfile.delimiter'='|'
        
        schema: 
        id	INT4
        name	TEXT
        score	FLOAT4
        type	TEXT
        
      • SQL
        select t1.id, t1.name, t1.cnt
        from (
          select a.id, a.name, count(*) as cnt
          from table1 a
          group by a.id, a.name
        ) t1
        where t1.cnt > 0
        
      • Error Message
        java.lang.NullPointerException
        	at org.apache.tajo.catalog.Column.equals(Column.java:115)
        	at java.util.ArrayList.indexOf(ArrayList.java:216)
        	at java.util.ArrayList.contains(ArrayList.java:199)
        	at java.util.AbstractCollection.containsAll(AbstractCollection.java:278)
        	at org.apache.tajo.catalog.Schema.containsAll(Schema.java:269)
        	at org.apache.tajo.engine.planner.LogicalPlanner.checkIfBeEvaluatedAtRelation(LogicalPlanner.java:1575)
        	at org.apache.tajo.engine.planner.rewrite.FilterPushDownRule.visitScan(FilterPushDownRule.java:315)
        	at org.apache.tajo.engine.planner.rewrite.FilterPushDownRule.visitScan(FilterPushDownRule.java:35)
        
        

        If I remove where clause, it will run successfully.

      1. TAJO-787_2.patch
        3 kB
        Jaehwa Jung
      2. TAJO-787.patch
        3 kB
        Jaehwa Jung

        Issue Links

          Activity

          Hide
          blrunner Jaehwa Jung added a comment -

          Created a review request against branch master in reviewboard.
          https://reviews.apache.org/r/20548/

          Show
          blrunner Jaehwa Jung added a comment - Created a review request against branch master in reviewboard. https://reviews.apache.org/r/20548/
          Hide
          blrunner Jaehwa Jung added a comment -

          I uploaded second patch.

          Show
          blrunner Jaehwa Jung added a comment - I uploaded second patch.
          Hide
          hyunsik Hyunsik Choi added a comment -

          +1

          I manually verified the patch. The patch looks good to me. Ship it.

          Show
          hyunsik Hyunsik Choi added a comment - +1 I manually verified the patch. The patch looks good to me. Ship it.
          Hide
          blrunner Jaehwa Jung added a comment -

          Thanks Hyunsik Choi.

          I've just committed it to master and 0.8.0 branch.

          Show
          blrunner Jaehwa Jung added a comment - Thanks Hyunsik Choi . I've just committed it to master and 0.8.0 branch.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-0.8.0-build #87 (See https://builds.apache.org/job/Tajo-0.8.0-build/87/)
          TAJO-787: FilterPushDownRule::visitSubQuery does not consider aliased columns. (jaehwa) (jhjung: rev b0ff0314cd81e9e974d02809deb41d4b37e90ae1)

          • CHANGES.txt
          • tajo-core/src/test/resources/results/TestSelectQuery/testWhereCond2.result
          • tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/rewrite/FilterPushDownRule.java
          • tajo-core/src/test/resources/queries/TestSelectQuery/testWhereCond2.sql
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.8.0-build #87 (See https://builds.apache.org/job/Tajo-0.8.0-build/87/ ) TAJO-787 : FilterPushDownRule::visitSubQuery does not consider aliased columns. (jaehwa) (jhjung: rev b0ff0314cd81e9e974d02809deb41d4b37e90ae1) CHANGES.txt tajo-core/src/test/resources/results/TestSelectQuery/testWhereCond2.result tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java tajo-core/src/main/java/org/apache/tajo/engine/planner/rewrite/FilterPushDownRule.java tajo-core/src/test/resources/queries/TestSelectQuery/testWhereCond2.sql
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #195 (See https://builds.apache.org/job/Tajo-master-build/195/)
          TAJO-787: FilterPushDownRule::visitSubQuery does not consider aliased columns. (jaehwa) (jhjung: rev 248879fee0038c89b77d0bf6271023e718410e41)

          • tajo-core/src/test/resources/queries/TestSelectQuery/testWhereCond2.sql
          • CHANGES.txt
          • tajo-core/src/test/resources/results/TestSelectQuery/testWhereCond2.result
          • tajo-core/src/main/java/org/apache/tajo/engine/planner/rewrite/FilterPushDownRule.java
          • tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #195 (See https://builds.apache.org/job/Tajo-master-build/195/ ) TAJO-787 : FilterPushDownRule::visitSubQuery does not consider aliased columns. (jaehwa) (jhjung: rev 248879fee0038c89b77d0bf6271023e718410e41) tajo-core/src/test/resources/queries/TestSelectQuery/testWhereCond2.sql CHANGES.txt tajo-core/src/test/resources/results/TestSelectQuery/testWhereCond2.result tajo-core/src/main/java/org/apache/tajo/engine/planner/rewrite/FilterPushDownRule.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java

            People

            • Assignee:
              blrunner Jaehwa Jung
              Reporter:
              blrunner Jaehwa Jung
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development