[DATAFU-172] Multimap tests fail on second run with Spark 3

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Assignee: Eyal Allweil
• Reporter: Eyal Allweil
• Created: 12/Jul/23 11:49
• Updated: 14/Jan/24 19:03
• Resolved: 21/Dec/23 13:43

Description

When running UdafTests in datafu-spark with Spark 3, three tests - mapmerge, multiarrayset all nulls, and multiarrayset max keys fail upon second run with this error:

 

org.apache.spark.sql.AnalysisException: Can not create the managed table('`default`.`mapmerge_table`'). The associated location('file:/Users/eyal/git/datafu/datafu-spark/spark-warehouse/mapmerge_table') already exists.;
 

 

Issue Links

• is required by: DATAFU-169 Support Spark 3.x