[HIVE-7250] Adaptive compression buffer size for wide tables in ORC - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.14.0
Fix Version/s: 0.14.0
Component/s: File Formats
Labels:
- orcfile

Description

If the input table is wide (in the order of 1000s), ORC compression buffer size overhead becomes significant causing OOM issues. To overcome this issue, buffer size should be adaptively chosen based on the available memory and the number of columns.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HIVE-7250.1.patch
18/Jun/14 01:53
374 kB
Prasanth Jayachandran
HIVE-7250.2.patch
18/Jun/14 07:07
372 kB
Prasanth Jayachandran
HIVE-7250.3.patch
18/Jun/14 23:59
372 kB
Prasanth Jayachandran
HIVE-7250.4.patch
19/Jun/14 01:47
382 kB
Prasanth Jayachandran
HIVE-7250.5.patch
19/Jun/14 18:52
384 kB
Prasanth Jayachandran

Issue Links

links to

Review Board

Activity

People

Assignee:: Prasanth Jayachandran

Reporter:: Prasanth Jayachandran

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 18/Jun/14 01:18

Updated:: 13/Nov/14 19:40

Resolved:: 20/Jun/14 02:32