Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4980

Drop partition against table with custom partition paths is failing with Error making 'dropPartition' RPC to Hive Metastore

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Not A Bug
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      100K-PARTITIONS-1M-FILES-CUSTOM-11-DROP-PARTITION
      <class 'impala._thrift_gen.beeswax.ttypes.BeeswaxException'> ImpalaRuntimeException: Error making 'dropPartition' RPC to Hive Metastore: CAUSED BY: MetaException: Got exception: java.io.IOException Failed to move to trash: hdfs://foo:8020/user/hive/warehouse/metadata_benchmarks_new_part.db/part=custom_part

      Repro steps

      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-04-INVALIDATE
      invalidate metadata 100k_partitions_1m_files_custom;
      ---- RESULTS
      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-05-QUERY-AFTER-INV
      select * from 100k_partitions_1m_files_custom limit 0;
      ---- RESULTS
      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-06-INVALIDATE
      invalidate metadata 100k_partitions_1m_files_custom;
      ---- RESULTS
      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-07-REFRESH
      REFRESH 100k_partitions_1m_files_custom;
      ---- RESULTS
      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-08-ADD-PARTITION
      alter table 100k_partitions_1m_files_custom add partition (part="custom_part")  location '/user/hive/warehouse/metadata_benchmarks_new_part.db/part=custom_part/';
      ---- RESULTS
      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-09-INVALIDATE
      invalidate metadata 100k_partitions_1m_files_custom;
      ---- RESULTS
      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-10-REFRESH-AFTER-ADD-PARTITION
      REFRESH 100k_partitions_1m_files_custom;
      ---- RESULTS
      ====
      ---- QUERY: 100K-PARTITIONS-1M-FILES-CUSTOM-11-DROP-PARTITION
      alter table 100k_partitions_1m_files_custom drop partition (part="custom_part");
      ---- RESULTS
      

        Activity

        Hide
        bharathv bharath v added a comment -

        Do you have the full hdfs stack trace from the HMS log?

        Show
        bharathv bharath v added a comment - Do you have the full hdfs stack trace from the HMS log?
        Hide
        alex.behm Alexander Behm added a comment -

        Permissions issue wit the HDFS trash.

        Show
        alex.behm Alexander Behm added a comment - Permissions issue wit the HDFS trash.

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            mmokhtar Mostafa Mokhtar
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development