[HIVE-2907] Hive error when dropping a table with large number of partitions - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 0.9.0
Fix Version/s: 0.10.0
Component/s: Metastore
Labels:
- TODOC10
Environment:

General. Hive Metastore bug.

Description

Running into an "Out Of Memory" error when trying to drop a table with 128K partitions.

The methods dropTable in metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
and dropTable in ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java encounter out of memory errors
when dropping tables with lots of partitions because they try to load the metadata for every partition into memory.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HIVE-2907.1.patch.txt
26/Mar/12 18:36
28 kB
Mousom Dhar Gupta
ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.1.patch
26/Mar/12 19:00
28 kB
Phabricator
ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.2.patch
26/Mar/12 21:32
30 kB
Phabricator
ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.3.patch
26/Mar/12 21:38
30 kB
Phabricator
ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.4.patch
26/Mar/12 23:24
31 kB
Phabricator
ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.5.patch
27/Mar/12 17:40
31 kB
Phabricator
ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.6.patch
31/Mar/12 04:34
32 kB
Phabricator
HIVE-2907.2.patch.txt
03/Apr/12 16:46
32 kB
Mousom Dhar Gupta
HIVE-2907.3.patch.txt
03/Apr/12 16:49
32 kB
Mousom Dhar Gupta
ASF.LICENSE.NOT.GRANTED--HIVE-2907.D2505.7.patch
03/Apr/12 17:02
32 kB
Phabricator

Activity

People

Assignee:: Mousom Dhar Gupta

Reporter:: Mousom Dhar Gupta

Votes:: 0 Vote for this issue

Watchers:: 11 Start watching this issue

Dates

Created:: 26/Mar/12 18:18

Updated:: 03/Oct/15 07:56

Resolved:: 10/Apr/12 16:56

Time Tracking

Estimated:

10h

Remaining:

10h

Logged:

Not Specified