[SPARK-24521] Fix ineffective test in CachedTableSuite

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.3.1
• Fix Version/s: 2.3.2, 2.4.0
• Component/s: SQL
• Assignee: Li Jin
• Reporter: Li Jin
• Created: 11/Jun/18 20:07
• Updated: 19/Jun/18 17:43
• Resolved: 19/Jun/18 17:43

Description

test("withColumn doesn't invalidate cached dataframe") in CachedTableSuite doesn't not work because:

The UDF is executed and test count incremented when "df.cache()" is called and the subsequent "df.collect()" has no effect on the test result. 

link:

https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala#L86

Issue Links

• links to: [Github] Pull Request #21531 (icexelloss)