[HIVE-12427] HiveServer2: Improve HiveServer2 JDBC/ODBC ResultSet performance - part1

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: HiveServer2, JDBC, ODBC
• Assignee: Unassigned
• Reporter: Vaibhav Gumashta
• Created: 16/Nov/15 23:28
• Updated: 16/Aug/16 19:33

Description

The current implementation does a lot of CPU intensive work in HiveServer2. It deserializes the temporary results written to disk and also serializes the results into thrift objects in HS2. This adds to the latency of fetching results via HS2. Doing that work in the task nodes would get that work done in parallel and scale it, thereby reducing the time it takes to retrieve large results.

Issue Links

• relates to: HIVE-14549 HiveServer2: Improve HiveServer2 JDBC/ODBC ResultSet performance - part2

Sub-Tasks

• HiveServer2: Refactor/repackage HiveServer2's Thrift code so that it can be used in the tasks: Closed
• HiveServer2: Provide an option for HiveServer2 to stream serialized thrift results when they are available: Resolved
• HiveServer2: Provide an option to write serialized thrift objects in final tasks: Closed
• Empty resultset run into Exception when using Thrift Binary Serde: Closed
• bump a new api version for ThriftJDBCBinarySerde changes: Closed
• Executing join query on type Float using Thrift Serde will result in Float cast to Double error: Closed
• Update protocol version in TOpenSessionReq and TOpenSessionResp: Closed
• HiveServer2: Make the usage of server with JDBC thirft serde enabled, backward compatible for older clients: Closed
• HiveServer2: Evaluate if ThriftJDBCBinarySerde should implement VectorizedSerde: Resolved
• HiveServer2: Performance instrumentation for HIVE-12049 (serializing thrift ResultSets in tasks): Closed
• HiveServer2: Cleanup code which checks for ThriftJDBCSerde usage: Closed
• HiveServer2: Use user supplied fetch size to determine #rows serialized in tasks: Resolved
• HiveServer2: enable ThriftJDBCBinarySerde use by default: Patch Available