Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33828 SQL Adaptive Query Execution QA
  3. SPARK-33850

EXPLAIN FORMATTED doesn't show the plan for subqueries if AQE is enabled

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0, 3.0.1, 3.1.0, 3.2.0
    • 3.1.0
    • SQL
    • None

    Description

      When AQE is enabled, EXPLAIN FORMATTED doesn't show the plan for subqueries.

      spark.conf.set("spark.sql.adaptive.enabled", "true")
      val df = spark.range(1, 100)
      df.createTempView("df")
      spark.sql("SELECT (SELECT min(id) AS v FROM df)").explain("FORMATTED")
      
      == Physical Plan ==
      AdaptiveSparkPlan (3)
      +- Project (2)
       +- Scan OneRowRelation (1)
      
      
      (1) Scan OneRowRelation
      Output: []
      Arguments: ParallelCollectionRDD[0] at explain at <console>:24, OneRowRelation, UnknownPartitioning(0)
      
      (2) Project
      Output [1]: [Subquery subquery#3, [id=#20] AS scalarsubquery()#5L]
      Input: []
      
      (3) AdaptiveSparkPlan
      Output [1]: [scalarsubquery()#5L]
      Arguments: isFinalPlan=false
      

      Attachments

        Activity

          People

            sarutak Kousuke Saruta
            sarutak Kousuke Saruta
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: