[HIVE-18512] Get Results ReadAhead

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.4.0, 3.0.0
• Component/s: HiveServer2
• Assignee: Unassigned
• Reporter: David Mollitor
• Created: 22/Jan/18 19:19
• Updated: 30/Jul/21 21:42

Description

I don't have any data to back this up, but I wanted to put it on the radar.

It may be possible to improve performance of HS2 with an HDFS read-ahead reader for result data.  This would require adding a cache (configurable size) to the Driver/Context object and adding a separate thread for loading results asynchronously while the client is processing its current batch of results.  It seems that currently, results are loaded on demand.

https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/Driver.java#L2298