[HIVE-10629] Dropping table in an encrypted zone does not drop warehouse directory

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.2.1
• Component/s: Security
• Assignee: Eugene Koifman
• Reporter: Deepesh Khandelwal
• Created: 06/May/15 18:11
• Updated: 13/Jul/15 18:35
• Resolved: 20/May/15 17:16

Description

Drop table in an encrypted zone removes the table but not its data. The client sees the following on Hive CLI:

hive> drop table testtbl;
OK
Time taken: 0.158 seconds

On the Hive Metastore log following error is thrown:

2015-05-05 08:55:27,665 ERROR [pool-6-thread-142]: hive.log (MetaStoreUtils.java:logAndThrowMetaException(1200)) - Got exception: java.io.IOException Failed to move to trash: hdfs://node-1.example.com:8020/apps/hive/warehouse/encdb1.db/testtbl
java.io.IOException: Failed to move to trash: hdfs://node-1.example.com:8020/apps/hive/warehouse/encdb1.db/testtbl
        at org.apache.hadoop.fs.TrashPolicyDefault.moveToTrash(TrashPolicyDefault.java:160)
        at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:114)
        at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:95)
        at org.apache.hadoop.hive.shims.Hadoop23Shims.moveToAppropriateTrash(Hadoop23Shims.java:270)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreFsImpl.deleteDir(HiveMetaStoreFsImpl.java:47)
        at org.apache.hadoop.hive.metastore.Warehouse.deleteDir(Warehouse.java:229)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.deleteTableData(HiveMetaStore.java:1584)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1552)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_with_environment_context(HiveMetaStore.java:1705)
        at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
        at com.sun.proxy.$Proxy13.drop_table_with_environment_context(Unknown Source)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:9256)
        ....

The client should throw the error and maybe fail the drop table call. To delete the table data one currently has to use drop table testtbl purge which basically remove the table data permanently skipping trash.

Issue Links

• is related to: HIVE-10986 Check of fs.trash.interval in HiveMetaStore should be consistent with Trash.moveToAppropriateTrash()
• relates to: HIVE-10747 Enable the cleanup of side effect for the Encryption related qfile test
• relates to: HIVE-10717 Fix failed qtest encryption_insert_partition_static test in Jenkin
• requires: HIVE-10978 Document fs.trash.interval wrt Hive and HDFS Encryption