Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22971

Eliminate file rename in insert-only compactor

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • None

    Description

      File rename is expensive for object stores, so MM (insert-only) compaction should skip that step when committing and write directly to base_x_cZ or delta_x_y_cZ.

      This also fixes the issue that for MM QB compaction the temp tables were stored under the table directory, and these temp dirs were never cleaned up.

      Attachments

        1. HIVE-22971.01.patch
          12 kB
          Karen Coppage
        2. HIVE-22971.02.patch
          13 kB
          Karen Coppage
        3. HIVE-22971.03.patch
          48 kB
          Karen Coppage
        4. HIVE-22971.04.patch
          38 kB
          Karen Coppage
        5. HIVE-22971.05.patch
          39 kB
          Karen Coppage
        6. HIVE-22971.06.patch
          39 kB
          Karen Coppage
        7. HIVE-22971.06.patch
          39 kB
          Karen Coppage
        8. HIVE-22971.06.patch
          39 kB
          Karen Coppage
        9. HIVE-22971.06.patch
          39 kB
          Karen Coppage

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-20948 Eliminate file rename in compactor

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link review board

          Activity

            People

              klcopp Karen Coppage
              klcopp Karen Coppage
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: