[SPARK-20908] Cache Manager: Hint should be ignored in plan matching

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.1, 2.2.0
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Xiao Li
• Reporter: Xiao Li
• Created: 28/May/17 00:31
• Updated: 28/May/17 04:34
• Resolved: 28/May/17 04:34

Description

In Cache manager, the plan matching should ignore Hint.

      val df1 = spark.range(10).join(broadcast(spark.range(10)))
      df1.cache()
      spark.range(10).join(spark.range(10)).explain()

The output plan of the above query shows that the second query is not using the cached data of the first query.

BroadcastNestedLoopJoin BuildRight, Inner
:- *Range (0, 10, step=1, splits=2)
+- BroadcastExchange IdentityBroadcastMode
   +- *Range (0, 10, step=1, splits=2)

Issue Links

• links to: [Github] Pull Request #18131 (gatorsmile)