[SPARK-5918] Spark Thrift server reports metadata for VARCHAR column as STRING in result set schema

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.1.1, 1.2.0
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Cheng Lian
• Reporter: Holman Lan
• Created: 20/Feb/15 04:16
• Updated: 21/May/19 05:36
• Resolved: 21/May/19 05:36

Description

This is reproducible using the open source JDBC driver by executing a query that will return a VARCHAR column then retrieving the result set metadata. The type name returned by the JDBC driver is VARCHAR which is expected but reports the column type as string[12] and precision/column length as 2147483647 (which is what the JDBC driver would return for STRING column) even though we created a VARCHAR column with max length of 1000.

Further investigation indicates the GetResultSetMetadata Thrift client API call returns the incorrect metadata.

We have confirmed this behaviour in versions 1.1.1 and 1.2.0. We have not yet tested this against 1.2.1 but will do so and report our findings.

Issue Links

• is duplicated by: SPARK-11660 Spark Thrift GetResultSetMetadata describes a VARCHAR as a STRING