[SPARK-3196] Expression Evaluation Performance Improvement

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Cheng Hao
• Created: 25/Aug/14 07:49
• Updated: 19/Dec/14 21:31
• Resolved: 19/Dec/14 21:31

Description

The expression id generations depend on a atomic long object internally, which will cause the performance drop dramatically in a multi-threading execution.

I'd like to create 2 sub tasks(maybe more) for the improvements:

1) Reduce the expression tree object creation from the aggregation functions (min/max), as they will create expression trees for each single row.
2) Remove the id property from the TreeNode API.

And remove the expression object creation as many as possible, where we have the expression evaluation. (I will create couple of subtask soon).

Sub-Tasks

• Reduce the expression tree object creation from the aggregation functions (min/max): Resolved
• Remove the id property from the TreeNode API: Resolved