Uploaded image for project: 'Tajo (Retired)'
  1. Tajo (Retired)
  2. TAJO-744 (Umbrella) ALTER TABLE ADD/DROP PARTITION statement
  3. TAJO-1865

When building filtered partition path, check partition directory exists on filesystem.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • Planner/Optimizer
    • None

    Description

      When building filtered partition path, Tajo need to check the path exists on file system. Currently, if the partition exists on catalog and it doesn't exist on file system, users would fail to execute a count all query as following:

      2015-09-17 15:54:50,219 ERROR org.apache.tajo.master.GlobalEngine: 
      Stack Trace:
      org.apache.tajo.exception.TajoInternalError: internal error: File does not exist: /tajo/warehouse/tpch_partition/customer/c_nationkey=9
      	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.getContentSummary(FSDirectory.java:2157)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:4398)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getContentSummary(NameNodeRpcServer.java:998)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getContentSummary(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
      
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.updateTableStat(PartitionedTableRewriter.java:439)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.access$100(PartitionedTableRewriter.java:49)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter$Rewriter.visitScan(PartitionedTableRewriter.java:523)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter$Rewriter.visitScan(PartitionedTableRewriter.java:508)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:110)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitGroupBy(BasicLogicalPlanVisitor.java:221)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:83)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:163)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:62)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.rewrite(PartitionedTableRewriter.java:82)
      

      Attachments

        Issue Links

          Activity

            People

              blrunner JaeHwa Jung
              blrunner JaeHwa Jung
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: