[SPARK-29013] Structurally equivalent subexpression elimination

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 3.0.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: L. C. Hsieh
• Created: 06/Sep/19 18:19
• Updated: 27/Dec/19 19:00
• Resolved: 27/Dec/19 19:00

Description

We do semantically equivalent subexpression elimination in SparkSQL. However, for some expressions that are not semantically equivalent, but structurally equivalent, current subexpression elimination generates too many similar functions. These functions share same computation structure but only differ in input slots of current processing row.

For such expressions, we can generate just one function, and pass in input slots during runtime.

It can reduce the length of generated code text, and save compilation time.

 

Issue Links

• links to: GitHub Pull Request #25717