[SPARK-30276] Support Filter expression allows simultaneous use of DISTINCT

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: SQL
• Assignee: Jiaan Geng
• Reporter: Jiaan Geng
• Created: 17/Dec/19 01:56
• Updated: 08/May/21 09:18
• Resolved: 04/Aug/20 04:41

Description

SPARK-27986 only supports  Filter expression without DISTINCT.

We need to support Filter expression allow simultaneous use of DISTINCT

PostgreSQL support:

select ten, sum(distinct four) filter (where four > 10) from onek group by ten;

Issue Links

• links to: [Github] Pull Request #29051 (beliefer)
• links to: [Github] Pull Request #29135 (beliefer)
• links to: [Github] Pull Request #29291 (beliefer)
• links to: GitHub Pull Request #27058
• links to: GitHub Pull Request #27428