[HIVE-1815] The class HiveResultSet should implement batch fetching.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.8.0
• Component/s: JDBC

Environment

Custom Java application using the Hive JDBC driver to connect to a Hive server, execute a Hive query and process the results.

• Hadoop Flags: Reviewed
• Release Note: Use batch fetching on the hive jdbc driver to increase performance.
• Assignee: Bennie Schut
• Reporter: Guy le Mar
• Created: 30/Nov/10 21:46
• Updated: 16/Dec/11 23:56
• Resolved: 18/Mar/11 04:16

Description

When using the Hive JDBC driver, you can execute a Hive query and obtain a HiveResultSet instance that contains the results of the query.
Unfortunately, HiveResultSet can then only fetch a single row of these results from the Hive server at a time. As a consequence, it's extremely slow to fetch a resultset of anything other than a trivial size.

It would be nice for the HiveResultSet to be able to fetch N rows from the server at a time, so that performance is suitable to support applications that provide human interaction.
(From memory, I think it took me around 20 minutes to fetch 4000 rows.)