Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32541

MSCK REPAIR TABLE should drop the missing partitions from metastore

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 3.0.0
    • None
    • SQL
    • None

    Description

      msck repair table <tablename> alter table <tablename> recover partitions is often used in environments where the new partitions are loaded as directories on HDFS and users want to create the missing partitions in bulk. However, currently it only supports addition of missing partitions. If there are any partitions which are present in metastore but not on the FileSystem, it should also delete them so that it truly repairs the table metadata.

      In Hive, it already support the grammar `msck repair table drop partitions`.

       

      https://issues.apache.org/jira/browse/HIVE-17824

      Attachments

        Activity

          People

            Unassigned Unassigned
            lipzhu Zhu, Lipeng
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: