[HIVE-4248] Implement a memory manager for ORC - ASF JIRA

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.11.0
Component/s: Serializers/Deserializers
Labels:
None

Description

With the large default stripe size (256MB) and dynamic partitions, it is quite easy for users to run out of memory when writing ORC files. We probably need a solution that keeps track of the total number of concurrent ORC writers and divides the available heap space between them.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HIVE-4248.D9993.1.patch
05/Apr/13 23:15
19 kB
Phabricator
HIVE-4248.D9993.2.patch
05/Apr/13 23:41
19 kB
Phabricator
HIVE-4248.D9993.4.patch
10/Apr/13 23:17
24 kB
Phabricator

Activity

People

Assignee:: Owen O'Malley

Reporter:: Owen O'Malley

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 28/Mar/13 21:46

Updated:: 09/Aug/14 19:31

Resolved:: 20/Apr/13 23:06