Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31997

Should drop test_udtf table when SingleSessionSuite completed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 2.4.7, 3.0.1, 3.1.0
    • Tests
    • None

    Description

      If we execute mvn test SingleSessionSuite and HiveThriftBinaryServerSuite in order,  the test case "SPARK-11595 ADD JAR with input path having URL scheme"  in HiveThriftBinaryServerSuite will failed as following:

       

      - SPARK-11595 ADD JAR with input path having URL scheme *** FAILED *** java.sql.SQLException: Error running query: org.apache.spark.sql.AnalysisException: Can not create the managed table('`default`.`test_udtf`'). The associated location('file:/home/yarn/spark_ut/spark_ut/baidu/inf-spark/spark-source/sql/hive-thriftserver/spark-warehouse/test_udtf') already exists.; at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:385) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:254) at org.apache.spark.sql.hive.thriftserver.HiveThriftBinaryServerSuite.$anonfun$new$65(HiveThriftServer2Suites.scala:603) at org.apache.spark.sql.hive.thriftserver.HiveThriftBinaryServerSuite.$anonfun$new$65$adapted(HiveThriftServer2Suites.scala:603) at scala.collection.immutable.List.foreach(List.scala:392) at org.apache.spark.sql.hive.thriftserver.HiveThriftBinaryServerSuite.$anonfun$new$63(HiveThriftServer2Suites.scala:603) at org.apache.spark.sql.hive.thriftserver.HiveThriftBinaryServerSuite.$anonfun$new$63$adapted(HiveThriftServer2Suites.scala:573) at org.apache.spark.sql.hive.thriftserver.HiveThriftJdbcTest.$anonfun$withMultipleConnectionJdbcStatement$3(HiveThriftServer2Suites.scala:1074) at org.apache.spark.sql.hive.thriftserver.HiveThriftJdbcTest.$anonfun$withMultipleConnectionJdbcStatement$3$adapted(HiveThriftServer2Suites.scala:1074) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
      

       

      because SingleSessionSuite do `create table test_udtf` and not drop it when test  completed and HiveThriftBinaryServerSuite want to re-create this table.

      If we execute mvn test HiveThriftBinaryServerSuite and SingleSessionSuite in order,both test suites will succeed, but we shouldn't rely on their execution order

       

      Attachments

        Activity

          People

            LuciferYang Yang Jie
            LuciferYang Yang Jie
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: