[SPARK-34952] DS V2 Aggregate push down

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: SQL
• Assignee: Huaxin Gao
• Reporter: Huaxin Gao
• Created: 04/Apr/21 06:38
• Updated: 07/Sep/21 17:43
• Resolved: 26/Jul/21 08:03

Description

Push down aggregate to data source for better performance. This will be done in two steps:
1. add aggregate push down APIs and the implementation in JDBC
2. add the implementation in Parquet.

Issue Links

• duplicates: SPARK-22390 Aggregate push down
• duplicates: SPARK-32833 JDBC V2 Datasource aggregate push down
• is depended upon by: SPARK-34960 Aggregate (Min/Max/Count) push down for ORC
• is duplicated by: SPARK-35511 Spark computes all rows during count() on a parquet file
• links to: [Github] Pull Request #32049 (huaxingao)
• links to: [Github] Pull Request #33352 (huaxingao)
• links to: [Github] Pull Request #33352 (huaxingao)
• links to: [Github] Pull Request #33526 (huaxingao)
• links to: [Github] Pull Request #33526 (huaxingao)
• links to: [Github] Pull Request #33579 (cloud-fan)
• links to: [Github] Pull Request #33639 (huaxingao)
• links to: [Github] Pull Request #33639 (huaxingao)
• links to: [Github] Pull Request #33739 (huaxingao)
• links to: [Github] Pull Request #33815 (huaxingao)
• links to: [Github] Pull Request #33815 (huaxingao)
• links to: [Github] Pull Request #33927 (huaxingao)
• links to: [Github] Pull Request #33927 (huaxingao)