[HIVE-20279] HiveContextAwareRecordReader slows down Druid Scan queries. - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 4.0.0-alpha-1
Component/s: None
Labels:
None

Description

HiveContextAwareRecordReader add lots of overhead for Druid Scan Queries.
See attached flame graph.
Looks like the operations for checking for existence of footer/header buffer takes most of time For druid and other storage handlers that do not have footer buffer we should skip the logic for checking the existence for storage handlers atleast.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

scan2.svg
31/Jul/18 11:59
110 kB
Nishant Bangarwa
HIVE-20279.patch
07/Aug/18 18:25
3 kB
Nishant Bangarwa
HIVE-20279.2.patch
08/Aug/18 11:52
1 kB
Nishant Bangarwa
HIVE-20279.1.patch
07/Aug/18 18:45
3 kB
Nishant Bangarwa

Activity

People

Assignee:: Nishant Bangarwa

Reporter:: Nishant Bangarwa

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 31/Jul/18 12:00

Updated:: 17/Nov/22 08:54

Resolved:: 08/Aug/18 20:31