Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-469

CTAS with no column definition will get a NPE

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      If you do a CTAS like this way

      create table  tbl partition by column(key float8) as select l_orderkey, l_partkey, l_quantity as key from lineitem;
      

      You will get an NPE because that expr.getTableElements() returns null and this null value later be iterated.

      2013-12-30 23:28:55,582 ERROR master.GlobalEngine (GlobalEngine.java:executeQuery(177)) - 
      Stack Trace:
      java.lang.NullPointerException
      	at org.apache.tajo.engine.planner.LogicalPlanner.convertTableElementsSchema(LogicalPlanner.java:956)
      	at org.apache.tajo.engine.planner.LogicalPlanner.convertTableElementsPartition(LogicalPlanner.java:844)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitCreateTable(LogicalPlanner.java:785)
      	at org.apache.tajo.engine.planner.LogicalPlanner.visitCreateTable(LogicalPlanner.java:81)
      	at org.apache.tajo.engine.planner.BaseAlgebraVisitor.visit(BaseAlgebraVisitor.java:98)
      	at org.apache.tajo.engine.planner.LogicalPlanner.createPlan(LogicalPlanner.java:114)
      	at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:225)
      	at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:138)
      	at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.submitQuery(TajoMasterClientService.java:134)
      	at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:257)
      	at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:91)
      	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
      	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
      	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
      	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
      	at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
      	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
      	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
      	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
      	at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
      	at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
      	at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
      	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
      	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
      	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
      	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
      	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
      	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
      	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
      	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
      	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
      	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
      	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
      	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      	at java.lang.Thread.run(Thread.java:680)
      

      I think this is caused by one of the different between create table DDL and CTAS. We can legally create a table by CTAS without define each columns' schema, just can use the output schema of select clause. But, create table DDL can't do it.

        Activity

        Hide
        hyunsik Hyunsik Choi added a comment -

        +1
        I totally agree with you. CTAS without a table schema should work as you mentioned.

        Show
        hyunsik Hyunsik Choi added a comment - +1 I totally agree with you. CTAS without a table schema should work as you mentioned.
        Hide
        coderplay Min Zhou added a comment - - edited

        Currently, I check expr.getTableElements() is null or not. If it is null, just use the select clause output schema fill it. This is the minimum change approach.

        Show
        coderplay Min Zhou added a comment - - edited Currently, I check expr.getTableElements() is null or not. If it is null, just use the select clause output schema fill it. This is the minimum change approach.
        Hide
        hyunsik Hyunsik Choi added a comment -

        +1 for the patch.

        The solution is straightforward, and the patch includes an unit test to verify this fix.

        Show
        hyunsik Hyunsik Choi added a comment - +1 for the patch. The solution is straightforward, and the patch includes an unit test to verify this fix.
        Hide
        hyunsik Hyunsik Choi added a comment -

        committed the patch to master. Thank you for your contribution!

        Show
        hyunsik Hyunsik Choi added a comment - committed the patch to master. Thank you for your contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-trunk-postcommit #662 (See https://builds.apache.org/job/Tajo-trunk-postcommit/662/)
        TAJO-469: CTAS with no column definition will get a NPE. (Min Zhou via hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=7aba3958e2d7f3ae5f82cc50928f778566430b1c)

        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java
        • tajo-core/tajo-core-backend/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlanner.java
        • CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-trunk-postcommit #662 (See https://builds.apache.org/job/Tajo-trunk-postcommit/662/ ) TAJO-469 : CTAS with no column definition will get a NPE. (Min Zhou via hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=7aba3958e2d7f3ae5f82cc50928f778566430b1c ) tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java tajo-core/tajo-core-backend/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4 tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlanner.java CHANGES.txt

          People

          • Assignee:
            coderplay Min Zhou
            Reporter:
            coderplay Min Zhou
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development