Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20710

Support aliases in CUBE/ROLLUP/GROUPING SETS

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.1
    • 2.2.0
    • SQL
    • None

    Description

      The current master supports regular group-by aliases though, it does not support for CUBE/ROLLUP/GROUPING SETS.

      sql("""
      CREATE OR REPLACE TEMPORARY VIEW testData AS SELECT * FROM VALUES
      (1, 1), (1, 2), (2, 1), (2, 2), (3, 1), (3, 2)
      AS testData(a, b)
      """)

      sql("SELECT a + b, b AS k, SUM(a - b) FROM testData GROUP BY a + b, k GROUPING SETS(k)").show

      scala> sql("SELECT a + b, b AS k, SUM(a - b) FROM testData GROUP BY a + b, k GROUPING SETS(k)").show
      org.apache.spark.sql.AnalysisException: cannot resolve '`k`' given input columns: [a, b]; line 1 pos 79;
      'GroupingSets [ArrayBuffer('k)], (a#61 + b#62), 'k, (a#61 + b#62), b#62, sum(cast((a#61 - b#62) as bigint))
      +- SubqueryAlias testdata
      +- Project a#61, b#62
      +- SubqueryAlias testData
      +- LocalRelation a#61, b#62

      at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
      at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:88)
      at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:85)
      at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:289)
      at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:289)
      at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:70)
      at org.apache.spark.sql.catalyst.trees.TreeNode.transformUp(TreeNode.scala:288)
      at org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$transformExpressionsUp$1.apply(QueryPlan.scala:267)
      at org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$transformExpressionsUp$1.apply(QueryPlan.scala:267)
      at org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpression$1(QueryPlan.scal

      Attachments

        Activity

          People

            maropu Takeshi Yamamuro
            maropu Takeshi Yamamuro
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: