Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36275

ResolveAggregateFunctions should work with nested fields

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: SQL
    • Labels:
      None

      Description

      A sort after Aggregate can fail to resolve if it contains nested fields. For example

      SELECT c.x, SUM(c.y)
      FROM VALUES NAMED_STRUCT('x', 'A', 'y', 1), NAMED_STRUCT('x', 'A', 'y', 2) AS t(c)
      GROUP BY c.x
      ORDER BY c.x
      

      Error:

      org.apache.spark.sql.AnalysisException: cannot resolve 'c.x' given input columns: [sum(c.y), x]; line 5 pos 9;
      'Sort ['c.x ASC NULLS FIRST], true
      +- Aggregate [c#0.x], [c#0.x AS x#2, sum(c#0.y) AS sum(c.y)#5L]
         +- SubqueryAlias t
            +- LocalRelation [c#0]
      

       

        Attachments

          Activity

            People

            • Assignee:
              allisonwang-db Allison Wang
              Reporter:
              allisonwang-db Allison Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: