Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27439

Explainging Dataset should show correct resolved plans

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.1
    • 3.0.0
    • SQL
    • None

    Description

      scala> spark.range(10).createOrReplaceTempView("test")
      scala> spark.range(5).createOrReplaceTempView("test2")
      scala> spark.sql("select * from test").createOrReplaceTempView("tmp001")
      scala> val df = spark.sql("select * from tmp001")
      scala> spark.sql("select * from test2").createOrReplaceTempView("tmp001")
      scala> df.show
      +---+
      | id|
      +---+
      |  0|
      |  1|
      |  2|
      |  3|
      |  4|
      |  5|
      |  6|
      |  7|
      |  8|
      |  9|
      +---+
      scala> df.explain
      

      Before:

      == Physical Plan ==
      *(1) Range (0, 5, step=1, splits=12)
      

      After:

      == Physical Plan ==
      *(1) Range (0, 10, step=1, splits=12)
      

      Attachments

        Activity

          People

            viirya L. C. Hsieh
            lemont xjl
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: