Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2907

Hive error when dropping a table with large number of partitions

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.10.0
    • Component/s: Metastore
    • Labels:
    • Environment:

      General. Hive Metastore bug.

      Description

      Running into an "Out Of Memory" error when trying to drop a table with 128K partitions.

      The methods dropTable in metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
      and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java encounter out of memory errors
      when dropping tables with lots of partitions because they try to load the metadata for every partition into memory.

        Attachments

        1. HIVE-2907.3.patch.txt
          32 kB
          Mousom Dhar Gupta
        2. HIVE-2907.2.patch.txt
          32 kB
          Mousom Dhar Gupta
        3. HIVE-2907.1.patch.txt
          28 kB
          Mousom Dhar Gupta
        4. ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.7.patch
          32 kB
          Phabricator
        5. ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.6.patch
          32 kB
          Phabricator
        6. ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.5.patch
          31 kB
          Phabricator
        7. ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.4.patch
          31 kB
          Phabricator
        8. ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.3.patch
          30 kB
          Phabricator
        9. ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.2.patch
          30 kB
          Phabricator
        10. ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.1.patch
          28 kB
          Phabricator

          Activity

            People

            • Assignee:
              mousom Mousom Dhar Gupta
              Reporter:
              mousom Mousom Dhar Gupta
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 10h
                10h
                Remaining:
                Remaining Estimate - 10h
                10h
                Logged:
                Time Spent - Not Specified
                Not Specified