[SPARK-41635] GROUP BY ALL

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 20/Dec/22 23:01
• Updated: 06/Nov/23 13:29
• Resolved: 22/Dec/22 11:59

Description

This patch implements GROUP BY ALL, similar to the one initially implemented in DuckDB. When specified, the analyzer automatically infers the grouping columns based on the expressions specified in the select clause: all expressions that don't include any aggregate expressions are pulled implicitly into the grouping columns. This avoids users having to specify individually the list of grouping columns in most cases.

Examples: 

select key, count, sum(score) from table group by all
-- rewritten to
select key, count, sum(score) from table group by key

Issue Links

• is related to: SPARK-45806 GROUP BY ALL don't work in ansi mode
• is related to: SPARK-41637 ORDER BY ALL
• links to: [Github] Pull Request #39134 (rxin)
• links to: [Github] Pull Request #39134 (rxin)
• links to: [Github] Pull Request #39184 (rxin)
• links to: [Github] Pull Request #39509 (cloud-fan)
• links to: [Github] Pull Request #39509 (cloud-fan)
• links to: GitHub Pull Request #39134