Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13354

Add ability to specify Compaction options per table and per request

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 2.0.0
    • Fix Version/s: 1.3.0, 2.1.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently the are a few options that determine when automatic compaction is triggered. They are specified once for the warehouse.
      This doesn't make sense - some table may be more important and need to be compacted more often.
      We should allow specifying these on per table basis.

      Also, compaction is an MR job launched from within the metastore. There is currently no way to control job parameters (like memory, for example) except to specify it in hive-site.xml for metastore which means they are site wide.

      Should add a way to specify these per table (perhaps even per compaction if launched via ALTER TABLE)

        Attachments

        1. HIVE-13354.branch-1.patch
          635 kB
          Wei Zheng
        2. HIVE-13354.3.patch
          768 kB
          Wei Zheng
        3. HIVE-13354.2.patch
          767 kB
          Wei Zheng
        4. HIVE-13354.1.withoutSchemaChange.patch
          753 kB
          Wei Zheng
        5. HIVE-13354.1.patch
          769 kB
          Wei Zheng

          Issue Links

            Activity

              People

              • Assignee:
                wzheng Wei Zheng
                Reporter:
                ekoifman Eugene Koifman
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: