Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33292

Make Literal ArrayBasedMapData string representation disambiguous

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2, 2.1.3, 2.2.3, 2.3.4, 2.4.7, 3.0.1, 3.1.0
    • Fix Version/s: 2.4.8, 3.0.2, 3.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Literal ArrayBasedMapData has inconsistent string representation from `LogicalPlan` to `Optimized Logical Plan/Physical Plan`. Also, the representation at `Optimized Logical Plan` and `Physical Plan` is ambiguous like `keys: [key1], values: [value1] AS c#0`.

      scala> spark.version
      res0: String = 2.4.7
      
      scala> sql("SELECT map('key1', 'value1') c").explain(true)
      == Parsed Logical Plan ==
      'Project ['map(key1, value1) AS c#0]
      +- OneRowRelation
      
      == Analyzed Logical Plan ==
      c: map<string,string>
      Project [map(key1, value1) AS c#0]
      +- OneRowRelation
      
      == Optimized Logical Plan ==
      Project [keys: [key1], values: [value1] AS c#0]
      +- OneRowRelation
      
      == Physical Plan ==
      *(1) Project [keys: [key1], values: [value1] AS c#0]
      +- Scan OneRowRelation[]
      

        Attachments

          Activity

            People

            • Assignee:
              dongjoon Dongjoon Hyun
              Reporter:
              dongjoon Dongjoon Hyun
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: