Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-580

Union query with partitioned tables cause NPE.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      I found a bug when I tested union query with partitioned tables as follows:

      create external table table1 (id int, name text, score float, type text) using csv with ('csvfile.delimiter'='|') location 'hdfs://localhost:9010/tajo/warehouse/table1' ;
      
      create table table2 (id int, name text, score float, type text) using csv with ('csvfile.delimiter'='|') partition by column(part text);
      
      insert overwrite into table2
      select id, name, score, type, '20140203'
      from table1;
      
      create table table3 (id int, name text, score float, type text) using csv with ('csvfile.delimiter'='|') partition by column(part text);
      
      select id, name 
      from table2 
      where part = '20140203' 
      union 
      select id, name 
      from table3 
      where part = '20140203';
      
      java.lang.NullPointerException
      	at org.apache.tajo.engine.planner.LogicalPlan.getNormalizedColumnName(LogicalPlan.java:183)
      	at org.apache.tajo.engine.planner.ExprNormalizer.visitColumnReference(ExprNormalizer.java:253)
      	at org.apache.tajo.engine.planner.ExprNormalizer.visitColumnReference(ExprNormalizer.java:77)
      	at org.apache.tajo.engine.planner.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:196)
      	at org.apache.tajo.engine.planner.SimpleAlgebraVisitor.visit(SimpleAlgebraVisitor.java:41)
      	at org.apache.tajo.engine.planner.SimpleAlgebraVisitor.visitBinaryOperator(SimpleAlgebraVisitor.java:56)
      	at org.apache.tajo.engine.planner.ExprNormalizer.visitBinaryOperator(ExprNormalizer.java:160)
      	at org.apache.tajo.engine.planner.ExprNormalizer.visitBinaryOperator(ExprNormalizer.java:77)
      	at org.apache.tajo.engine.planner.SimpleAlgebraVisitor.visit(SimpleAlgebraVisitor.java:38)
      	at org.apache.tajo.engine.planner.ExprNormalizer.normalize(ExprNormalizer.java:103)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitFilter(LogicalPlanner.java:701)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitFilter(LogicalPlanner.java:55)
      	at org.apache.tajo.engine.planner.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:73)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitProjection(LogicalPlanner.java:188)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitProjection(LogicalPlanner.java:55)
      	at org.apache.tajo.engine.planner.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:55)
      	at org.apache.tajo.engine.planner.LogicalPlanner.buildSetPlan(LogicalPlanner.java:1060)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitUnion(LogicalPlanner.java:1032)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitUnion(LogicalPlanner.java:55)
      	at org.apache.tajo.engine.planner.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:76)
      	at org.apache.tajo.engine.planner.LogicalPlanner.createPlan(LogicalPlanner.java:106)
      	at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:245)
      	at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:136)
      
      

      For reference, I used data file as follows:

      data.csv
      1|abc|1.1|a
      2|def|2.3|b
      3|ghi|3.4|c
      4|jkl|4.5|d
      5|mno|5.6|e
      

      And if I don't write where clause, I can't find this error.

        Issue Links

          Activity

          Hide
          blrunner Jaehwa Jung added a comment -

          It doesn't happen no more after applying TAJO-581.

          Show
          blrunner Jaehwa Jung added a comment - It doesn't happen no more after applying TAJO-581 .
          Hide
          hyunsik Hyunsik Choi added a comment -

          Hi Jaehwa,

          This bug is essentially the same to TAJO-581. I've just submitted the patch. Please take a look at the patch.

          Show
          hyunsik Hyunsik Choi added a comment - Hi Jaehwa, This bug is essentially the same to TAJO-581 . I've just submitted the patch. Please take a look at the patch.

            People

            • Assignee:
              blrunner Jaehwa Jung
              Reporter:
              blrunner Jaehwa Jung
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development