Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20707

Automatic partition management

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.0.0
    • 4.0.0-alpha-1
    • None
    • None

    Description

      In current scenario, to add partitions for external tables to metastore, MSCK REPAIR command has to be executed manually. To avoid this manual step, external tables can be specified a table property based on which a background metastore thread can sync partitions periodically. Tables can also be specified with partition retention period. Any partition whose age exceeds the retention period will be dropped automatically.

      Attachments

        1. HIVE-20707-branch-3.patch
          316 kB
          Prasanth Jayachandran
        2. HIVE-20707.7.patch
          308 kB
          Prasanth Jayachandran
        3. HIVE-20707.6.patch
          376 kB
          Prasanth Jayachandran
        4. HIVE-20707.6.patch
          376 kB
          Prasanth Jayachandran
        5. HIVE-20707.5.patch
          306 kB
          Prasanth Jayachandran
        6. HIVE-20707.4.patch
          306 kB
          Prasanth Jayachandran
        7. HIVE-20707.2.patch
          304 kB
          Prasanth Jayachandran
        8. HIVE-20707.1.patch
          223 kB
          Prasanth Jayachandran
        9. HIVE-20702.3.patch
          303 kB
          Prasanth Jayachandran

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-21072 NPE when running partitioned CTAS statements

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-25039 Disable discovery.partitions config for external tables by default

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-21103 PartitionManagementTask should not modify DN configs to avoid closing persistence manager

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-21482 Partition discovery table property is added to non-partitioned external tables

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-21582 Prefix msck configs with metastore

          • Major - Major loss of function.
          • Closed
          links to

          Web Link RB

          Activity

            People

              prasanth_j Prasanth Jayachandran
              prasanth_j Prasanth Jayachandran
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: