[IMPALA-2310] Add PURGE option to DROP TABLE

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.1
• Fix Version/s: Impala 2.3.0
• Labels: impala security usability
• Environment: Kerberos enabled cluster with HDFS Encryption
• Assignee: Bharath Vissapragada
• Reporter: Martin Nash
• Created: 08/Sep/15 10:22
• Updated: 28/Sep/16 21:16
• Resolved: 14/Oct/15 05:52

Description

Prior to using HDFS Encryption to create an encryption zone that is used to hold data files for an Impala table we would see the expected behaviour of DROP TABLE removing the data files for the table. Now that encryption is used the drop table command succeeds, however, the data files are not removed and the Hive Metastore log shows "can't be moved from an encryption zone."

Attempting to remove the data files using "hdfs dfs -rm" produces the same message as above, however, adding -skipTrash allows the files to be removed.

On the basis of the above, adding a PURGE option to DROP TABLE (and ALTER TABLE ... DROP PARTITION) that results in -skipTrash been used for the deletion of the data files would be very useful.

Issue Links

• is related to: IMPALA-2452 Enable DROP PURGE when it is available in Hive for Impala