Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-744 (Umbrella) ALTER TABLE ADD/DROP PARTITION statement
  3. TAJO-1865

When building filtered partition path, check partition directory exists on filesystem.

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      When building filtered partition path, Tajo need to check the path exists on file system. Currently, if the partition exists on catalog and it doesn't exist on file system, users would fail to execute a count all query as following:

      2015-09-17 15:54:50,219 ERROR org.apache.tajo.master.GlobalEngine: 
      Stack Trace:
      org.apache.tajo.exception.TajoInternalError: internal error: File does not exist: /tajo/warehouse/tpch_partition/customer/c_nationkey=9
      	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.getContentSummary(FSDirectory.java:2157)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:4398)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getContentSummary(NameNodeRpcServer.java:998)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getContentSummary(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
      
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.updateTableStat(PartitionedTableRewriter.java:439)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.access$100(PartitionedTableRewriter.java:49)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter$Rewriter.visitScan(PartitionedTableRewriter.java:523)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter$Rewriter.visitScan(PartitionedTableRewriter.java:508)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:110)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitGroupBy(BasicLogicalPlanVisitor.java:221)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:83)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:163)
      	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:62)
      	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.rewrite(PartitionedTableRewriter.java:82)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                blrunner Jaehwa Jung
                Reporter:
                blrunner Jaehwa Jung
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: