[SPARK-29492] SparkThriftServer can't support jar class as table serde class when executestatement in sync mode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: angerszhu
• Reporter: angerszhu
• Created: 16/Oct/19 12:57
• Updated: 29/Apr/20 06:49
• Resolved: 29/Apr/20 06:49

Description

Add UT in HiveThriftBinaryServerSuit:

  test("jar in sync mode") {
    withCLIServiceClient { client =>
      val user = System.getProperty("user.name")
      val sessionHandle = client.openSession(user, "")
      val confOverlay = new java.util.HashMap[java.lang.String, java.lang.String]
      val jarFile = HiveTestJars.getHiveHcatalogCoreJar().getCanonicalPath

      Seq(s"ADD JAR $jarFile",
        "CREATE TABLE smallKV(key INT, val STRING)",
        s"LOAD DATA LOCAL INPATH '${TestData.smallKv}' OVERWRITE INTO TABLE smallKV")
        .foreach(query => client.executeStatement(sessionHandle, query, confOverlay))

      client.executeStatement(sessionHandle,
        """CREATE TABLE addJar(key string)
          |ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
        """.stripMargin, confOverlay)

      client.executeStatement(sessionHandle,
        "INSERT INTO TABLE addJar SELECT 'k1' as key FROM smallKV limit 1", confOverlay)


      val operationHandle = client.executeStatement(
        sessionHandle,
        "SELECT key FROM addJar",
        confOverlay)

      // Fetch result first time
      assertResult(1, "Fetching result first time from next row") {

        val rows_next = client.fetchResults(
          operationHandle,
          FetchOrientation.FETCH_NEXT,
          1000,
          FetchType.QUERY_OUTPUT)

        rows_next.numRows()
      }
    }
  }

Run it then got ClassNotFound error.

Issue Links

• links to: GitHub Pull Request #26141