Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8065 Support HDFS encryption functionality on Hive
  3. HIVE-10629

Dropping table in an encrypted zone does not drop warehouse directory

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.1
    • Component/s: Security
    • Labels:
      None

      Description

      Drop table in an encrypted zone removes the table but not its data. The client sees the following on Hive CLI:

      hive> drop table testtbl;
      OK
      Time taken: 0.158 seconds
      

      On the Hive Metastore log following error is thrown:

      2015-05-05 08:55:27,665 ERROR [pool-6-thread-142]: hive.log (MetaStoreUtils.java:logAndThrowMetaException(1200)) - Got exception: java.io.IOException Failed to move to trash: hdfs://node-1.example.com:8020/apps/hive/warehouse/encdb1.db/testtbl
      java.io.IOException: Failed to move to trash: hdfs://node-1.example.com:8020/apps/hive/warehouse/encdb1.db/testtbl
              at org.apache.hadoop.fs.TrashPolicyDefault.moveToTrash(TrashPolicyDefault.java:160)
              at org.apache.hadoop.fs.Trash.moveToTrash(Trash.java:114)
              at org.apache.hadoop.fs.Trash.moveToAppropriateTrash(Trash.java:95)
              at org.apache.hadoop.hive.shims.Hadoop23Shims.moveToAppropriateTrash(Hadoop23Shims.java:270)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreFsImpl.deleteDir(HiveMetaStoreFsImpl.java:47)
              at org.apache.hadoop.hive.metastore.Warehouse.deleteDir(Warehouse.java:229)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.deleteTableData(HiveMetaStore.java:1584)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_core(HiveMetaStore.java:1552)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_table_with_environment_context(HiveMetaStore.java:1705)
              at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
              at com.sun.proxy.$Proxy13.drop_table_with_environment_context(Unknown Source)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_table_with_environment_context.getResult(ThriftHiveMetastore.java:9256)
              ....
      

      The client should throw the error and maybe fail the drop table call. To delete the table data one currently has to use drop table testtbl purge which basically remove the table data permanently skipping trash.

        Attachments

        1. HIVE-10629.patch
          8 kB
          Eugene Koifman
        2. HIVE-10629.5.patch
          32 kB
          Eugene Koifman
        3. HIVE-10629.4.patch
          8 kB
          Eugene Koifman
        4. HIVE-10629.3.patch
          8 kB
          Eugene Koifman
        5. HIVE-10629.2.patch
          8 kB
          Eugene Koifman

          Issue Links

            Activity

              People

              • Assignee:
                ekoifman Eugene Koifman
                Reporter:
                deepesh Deepesh Khandelwal
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: