Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20908

Cache Manager: Hint should be ignored in plan matching

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.1, 2.2.0
    • Fix Version/s: 2.2.0
    • Component/s: SQL
    • Labels:
      None

      Description

      In Cache manager, the plan matching should ignore Hint.

            val df1 = spark.range(10).join(broadcast(spark.range(10)))
            df1.cache()
            spark.range(10).join(spark.range(10)).explain()
      

      The output plan of the above query shows that the second query is not using the cached data of the first query.

      BroadcastNestedLoopJoin BuildRight, Inner
      :- *Range (0, 10, step=1, splits=2)
      +- BroadcastExchange IdentityBroadcastMode
         +- *Range (0, 10, step=1, splits=2)
      

        Attachments

          Activity

            People

            • Assignee:
              smilegator Xiao Li
              Reporter:
              smilegator Xiao Li
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: