[SPARK-16563] Repeat calling Spark SQL thrift server fetchResults return empty for ExecuteStatement operation

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.2
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: SQL
• Assignee: Gu Huiqin Alice
• Reporter: Gu Huiqin Alice
• Created: 15/Jul/16 05:58
• Updated: 31/Dec/16 11:52
• Resolved: 09/Aug/16 01:01

Description

Repeat calling FetchResults(... orientation=FetchOrientation.FETCH_FIRST ..) of spark sql thrift service will return empty set after calling ExecuteStatement of TCLIService.

The bug exist in function public RowSet getNextRowSet(FetchOrientation orientation, long maxRows)
https://github.com/apache/spark/blob/02c8072eea72425e89256347e1f373a3e76e6eba/sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/operation/SQLOperation.java#L332

The iterator for geting result can be used for only once, so repeat calling FetchResults with FETCH_FIRST parameter will return empty result.

FetchOrientation.FETCH_FIRST

Issue Links

• is related to: SPARK-18857 SparkSQL ThriftServer hangs while extracting huge data volumes in incremental collect mode
• links to: [Github] Pull Request #14218 (alicegugu)