[HIVE-22040] Drop partition throws exception with 'Failed to delete parent: File does not exist' when the partition's parent path does not exists

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Standalone Metastore
• Assignee: xiepengjie
• Reporter: xiepengjie
• Due: 31/Aug/19
• Created: 24/Jul/19 13:34
• Updated: 17/Nov/22 08:51
• Resolved: 06/Aug/19 20:41

Description

I create a manage table with multi partition columns, when i try to drop partition throws exception with 'Failed to delete parent: File does not exist' when the partition's parent path does not exist. The partition's metadata in mysql has been deleted, but the exception is still thrown. it will fail if  connecting hiveserver2 with jdbc by java, this problem also exists in master branch, I  think it is very unfriendly and we should fix it.

Example:

– First, create manage table with nulti partition columns, and add partitions:

drop table if exists t1;

create table t1 (c1 int) partitioned by (year string, month string, day string);

alter table t1 add partition(year='2019', month='07', day='01');

– Second, delete the path of partition 'month=07':

hadoop fs -rm -r /user/hadoop/xiepengjietest.db/drop_partition/year=2019/month=07

--  Third, when i try to drop partition, the metastore throws exception with 'Failed to delete parent: File does not exist' .

alter table t1 drop partition(year='2019', month='07', day='01');

exception like this:

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Failed to delete parent: File does not exist: /user/hadoop/xiepengjietest.db/drop_partition/year=2019/month=07
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getContentSummaryInt(FSDirStatAndListingOp.java:493)
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getContentSummary(FSDirStatAndListingOp.java:140)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:3995)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getContentSummary(NameNodeRpcServer.java:1202)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getContentSummary(ClientNamenodeProtocolServerSideTranslatorPB.java:883)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2115)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2111)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1867)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2111) (state=08S01,code=1)