Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5918

Spark Thrift server reports metadata for VARCHAR column as STRING in result set schema

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 1.1.1, 1.2.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:

      Description

      This is reproducible using the open source JDBC driver by executing a query that will return a VARCHAR column then retrieving the result set metadata. The type name returned by the JDBC driver is VARCHAR which is expected but reports the column type as string[12] and precision/column length as 2147483647 (which is what the JDBC driver would return for STRING column) even though we created a VARCHAR column with max length of 1000.

      Further investigation indicates the GetResultSetMetadata Thrift client API call returns the incorrect metadata.

      We have confirmed this behaviour in versions 1.1.1 and 1.2.0. We have not yet tested this against 1.2.1 but will do so and report our findings.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lian cheng Cheng Lian
                Reporter:
                holmanl Holman Lan
              • Votes:
                1 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: