[SPARK-32665] Deletes orphan directories under a warehouse dir in SQLQueryTestSuite

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: SQL, Tests
• Assignee: Takeshi Yamamuro
• Reporter: Takeshi Yamamuro
• Created: 20/Aug/20 08:12
• Updated: 20/Aug/20 13:12
• Resolved: 20/Aug/20 13:12

Description

In case that a last SQLQueryTestSuite test run is killed, it will fail in a next run because of a following reason:

[info] org.apache.spark.sql.SQLQueryTestSuite *** ABORTED *** (17 seconds, 483 milliseconds)
[info]   org.apache.spark.sql.AnalysisException: Can not create the managed table('`testdata`'). The associated location('file:/Users/maropu/Repositories/spark/spark-master/sql/core/spark-warehouse/org.apache.spark.sql.SQLQueryTestSuite/testdata') already exists.;
[info]   at org.apache.spark.sql.catalyst.catalog.SessionCatalog.validateTableLocation(SessionCatalog.scala:355)
[info]   at org.apache.spark.sql.execution.command.CreateDataSourceTableAsSelectCommand.run(createDataSourceTables.scala:170)
[info]   at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:108)

This ticket targets at adding code to deletes orphan directories under a warehouse dir in SQLQueryTestSuite before creating test tables.

Issue Links

• links to: [Github] Pull Request #29488 (maropu)
• links to: [Github] Pull Request #29488 (maropu)