[SPARK-48445] Don't inline UDFs with non-cheap children in CollapseProject

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.1
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Kelvin Jiang
• Reporter: Kelvin Jiang
• Created: 28/May/24 21:41
• Updated: 29/May/24 02:32
• Resolved: 29/May/24 02:32

Description

Because UDFs (and certain other expressions) are considered cheap by CollapseProject.isCheap, they are inlined and potentially duplicated (which is ok, because rules like ExtractPythonUDFs will de-duplicate them). However, if the UDFs contain other non-cheap expressions, those will also be duplicated and can potentially cause performance regressions.

Issue Links

• links to: GitHub Pull Request #46780