Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      The following query occurs NPE.

      select a.id, b.c_name from ( 
        select l_partkey as id from lineitem 
        union all 
        select o_orderkey as id from orders 
      ) a 
      join customer b on a.id = b.c_custkey
      
      2014-06-18 16:33:54,592 ERROR: org.apache.tajo.master.querymaster.QueryMasterTask (startQuery(364)) - 
      java.lang.NullPointerException
      	at org.apache.tajo.engine.planner.global.DataChannel.<init>(DataChannel.java:55)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner.createDataChannelFromJoin(GlobalPlanner.java:178)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner.buildJoinPlan(GlobalPlanner.java:315)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner.access$300(GlobalPlanner.java:54)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitJoin(GlobalPlanner.java:1106)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitJoin(GlobalPlanner.java:967)
      	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:84)
      	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:142)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitRoot(GlobalPlanner.java:972)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner$DistributedPlannerVisitor.visitRoot(GlobalPlanner.java:967)
      	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:58)
      	at org.apache.tajo.engine.planner.global.GlobalPlanner.build(GlobalPlanner.java:134)
      	at org.apache.tajo.master.querymaster.QueryMasterTask.startQuery(QueryMasterTask.java:356)
      	at org.apache.tajo.master.querymaster.QueryMasterTask.start(QueryMasterTask.java:176)
      	at org.apache.tajo.master.querymaster.QueryMaster$QueryStartEventHandler.handle(QueryMaster.java:380)
      	at org.apache.tajo.master.querymaster.QueryMaster$QueryStartEventHandler.handle(QueryMaster.java:371)
      	at org.apache.tajo.master.TajoAsyncDispatcher.dispatch(TajoAsyncDispatcher.java:137)
      	at org.apache.tajo.master.TajoAsyncDispatcher$1.run(TajoAsyncDispatcher.java:79)
      	at java.lang.Thread.run(Thread.java:744)
      
      

        Activity

        Hide
        hjkim Hyoungjun Kim added a comment - - edited

        I found the other error case that is a join with more than 3 tables in a union.

        select orders.o_orderkey 
        from orders
        join lineitem on orders.o_orderkey = lineitem.l_orderkey
        join customer on orders.o_custkey =  customer.c_custkey
        union all 
        select nation.n_nationkey from nation
        

        This query throws the following error.

        java.lang.NullPointerException
        	at org.apache.tajo.engine.planner.PlannerUtil.targetToSchema(PlannerUtil.java:626)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.verifyProjectableOutputSchema(LogicalPlanVerifier.java:62)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:134)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:31)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:84)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:128)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:31)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:84)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visitProjection(BasicLogicalPlanVisitor.java:152)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitProjection(LogicalPlanVerifier.java:84)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitProjection(LogicalPlanVerifier.java:31)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:63)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visitUnion(BasicLogicalPlanVisitor.java:226)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitUnion(LogicalPlanVerifier.java:165)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitUnion(LogicalPlanVerifier.java:31)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:87)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:142)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:58)
        	at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:45)
        	at org.apache.tajo.engine.planner.LogicalPlanVerifier.verify(LogicalPlanVerifier.java:52)
        	at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:491)
        	at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:148)
        

        I'll fix this error within this issue.

        Show
        hjkim Hyoungjun Kim added a comment - - edited I found the other error case that is a join with more than 3 tables in a union. select orders.o_orderkey from orders join lineitem on orders.o_orderkey = lineitem.l_orderkey join customer on orders.o_custkey = customer.c_custkey union all select nation.n_nationkey from nation This query throws the following error. java.lang.NullPointerException at org.apache.tajo.engine.planner.PlannerUtil.targetToSchema(PlannerUtil.java:626) at org.apache.tajo.engine.planner.LogicalPlanVerifier.verifyProjectableOutputSchema(LogicalPlanVerifier.java:62) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:134) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:31) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:84) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:128) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitJoin(LogicalPlanVerifier.java:31) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:84) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visitProjection(BasicLogicalPlanVisitor.java:152) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitProjection(LogicalPlanVerifier.java:84) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitProjection(LogicalPlanVerifier.java:31) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:63) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visitUnion(BasicLogicalPlanVisitor.java:226) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitUnion(LogicalPlanVerifier.java:165) at org.apache.tajo.engine.planner.LogicalPlanVerifier.visitUnion(LogicalPlanVerifier.java:31) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:87) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:142) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:58) at org.apache.tajo.engine.planner.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:45) at org.apache.tajo.engine.planner.LogicalPlanVerifier.verify(LogicalPlanVerifier.java:52) at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:491) at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:148) I'll fix this error within this issue.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user babokim opened a pull request:

        https://github.com/apache/tajo/pull/48

        TAJO-881: JOIN with union query occurs NPE

        Join with tableC and result of union tableA, tableB is expected the following physical plan. But Union execution block is not necessary.
        ```

        -eb_0001_000006 (Terminal)
        -eb_0001_000005 (Join eb_0001_000003, eb_0001_000004)
        -eb_0001_000004 (Scan TableC)
        -eb_0001_000003 (Union TableA, TableB)
        -eb_0001_000002 (Scan TableB)
        -eb_0001_000001 (Scan TableA)
        ```
        The above plan can be changed to the following plan.
        ```
        -eb_0001_000005 (Terminal)
        -eb_0001_000003 (Join [eb_0001_000001, eb_0001_000002], eb_0001_000004)
        -eb_0001_000004 (Scan TableC)
        -eb_0001_000002 (Scan TableB)
        -eb_0001_000001 (Scan TableA)
        ```
        eb_0001_000003's left child should be eb_0001_000001 + eb_0001_000001 and right child should be eb_0001_000004.
        For this eb_0001_000001 is representative of eb_0001_000001, eb_0001_000002.
        So eb_0001_000003's left child is eb_0001_000001

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/babokim/tajo TAJO-881

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/48.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #48


        commit 4d495e6f1ac29f255b08c4d2025e0d954b23e984
        Author: 김형준 <babokim@babokim-macbook-pro.local>
        Date: 2014-06-25T07:58:47Z

        TAJO-881: JOIN with union query occurs NPE

        commit 2e6f50939b26aecb33911fbcbfa2b912f708e0e8
        Author: 김형준 <babokim@babokim-macbook-pro.local>
        Date: 2014-06-25T07:59:36Z

        TAJO-881: JOIN with union query occurs NPE
        Remove unnecessary comment.

        commit aec5f9ade653b9c78b5a3c4b016f51561c04a126
        Author: 김형준 <babokim@babokim-macbook-pro.local>
        Date: 2014-06-25T08:01:56Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-881

        commit 04b4854f86996475250a4c96f7069169187c62f0
        Author: 김형준 <babokim@babokim-macbook-pro.local>
        Date: 2014-06-25T13:35:06Z

        TAJO-881: JOIN with union query occurs NPE
        Add some comment


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user babokim opened a pull request: https://github.com/apache/tajo/pull/48 TAJO-881 : JOIN with union query occurs NPE Join with tableC and result of union tableA, tableB is expected the following physical plan. But Union execution block is not necessary. ``` -eb_0001_000006 (Terminal) -eb_0001_000005 (Join eb_0001_000003, eb_0001_000004) -eb_0001_000004 (Scan TableC) -eb_0001_000003 (Union TableA, TableB) -eb_0001_000002 (Scan TableB) -eb_0001_000001 (Scan TableA) ``` The above plan can be changed to the following plan. ``` -eb_0001_000005 (Terminal) -eb_0001_000003 (Join [eb_0001_000001, eb_0001_000002] , eb_0001_000004) -eb_0001_000004 (Scan TableC) -eb_0001_000002 (Scan TableB) -eb_0001_000001 (Scan TableA) ``` eb_0001_000003's left child should be eb_0001_000001 + eb_0001_000001 and right child should be eb_0001_000004. For this eb_0001_000001 is representative of eb_0001_000001, eb_0001_000002. So eb_0001_000003's left child is eb_0001_000001 You can merge this pull request into a Git repository by running: $ git pull https://github.com/babokim/tajo TAJO-881 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/48.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #48 commit 4d495e6f1ac29f255b08c4d2025e0d954b23e984 Author: 김형준 <babokim@babokim-macbook-pro.local> Date: 2014-06-25T07:58:47Z TAJO-881 : JOIN with union query occurs NPE commit 2e6f50939b26aecb33911fbcbfa2b912f708e0e8 Author: 김형준 <babokim@babokim-macbook-pro.local> Date: 2014-06-25T07:59:36Z TAJO-881 : JOIN with union query occurs NPE Remove unnecessary comment. commit aec5f9ade653b9c78b5a3c4b016f51561c04a126 Author: 김형준 <babokim@babokim-macbook-pro.local> Date: 2014-06-25T08:01:56Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-881 commit 04b4854f86996475250a4c96f7069169187c62f0 Author: 김형준 <babokim@babokim-macbook-pro.local> Date: 2014-06-25T13:35:06Z TAJO-881 : JOIN with union query occurs NPE Add some comment
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/48#issuecomment-47184088

        +1

        The patch looks good to me and includes enough unit tests.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/48#issuecomment-47184088 +1 The patch looks good to me and includes enough unit tests.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/48#issuecomment-47184119

        Before commit it, I've renamed some methods and added more comments.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/48#issuecomment-47184119 Before commit it, I've renamed some methods and added more comments.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/48

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/48
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #266 (See https://builds.apache.org/job/Tajo-master-build/266/)
        TAJO-881: JOIN with union query occurs NPE. (Hyoungjun Kim via hyunsik) (hyunsik: rev 6cfd448f7fb60a254a237230f071456f188f9179)

        • tajo-core/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
        • tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
        • tajo-core/src/main/java/org/apache/tajo/master/DefaultTaskScheduler.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/enforce/Enforcer.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/rewrite/ProjectionPushDownRule.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/ExecutionBlock.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/MasterPlan.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #266 (See https://builds.apache.org/job/Tajo-master-build/266/ ) TAJO-881 : JOIN with union query occurs NPE. (Hyoungjun Kim via hyunsik) (hyunsik: rev 6cfd448f7fb60a254a237230f071456f188f9179) tajo-core/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java tajo-core/src/main/java/org/apache/tajo/master/DefaultTaskScheduler.java tajo-core/src/main/java/org/apache/tajo/engine/planner/enforce/Enforcer.java tajo-core/src/main/java/org/apache/tajo/engine/planner/rewrite/ProjectionPushDownRule.java CHANGES tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/ExecutionBlock.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/MasterPlan.java
        Hide
        hyunsik Hyunsik Choi added a comment -

        committed.

        Show
        hyunsik Hyunsik Choi added a comment - committed.

          People

          • Assignee:
            hjkim Hyoungjun Kim
            Reporter:
            hjkim Hyoungjun Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development