[SPARK-39651] Prune filter condition if compare with rand is deterministic

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: Jiaan Geng
• Reporter: Jiaan Geng
• Created: 01/Jul/22 04:27
• Updated: 13/Jul/22 06:06
• Resolved: 13/Jul/22 06:06

Description

The SQL

SELECT * FROM tab WHERE rand(1) < 2

evaluate rand(1) < 2 for rows one by one.
In fact, we can prune the filter condition.

Issue Links

• links to: [Github] Pull Request #37040 (beliefer)