[SPARK-11873] Regression for TPC-DS query 63 when used with decimal datatype and windows function

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Auto Closed
• Affects Version/s: 1.5.0
• Component/s: SQL
• Labels: perfomance
• Assignee: Unassigned
• Reporter: Dileep Kumar
• Created: 20/Nov/15 00:32
• Updated: 01/Jun/16 06:20
• Resolved: 01/Jun/16 06:20

Description

When running the TPC-DS based queries for benchmarking spark found that query 63 (after making it similar to original query) show different behavior compared to other queries eg. q98 which has similar function.
Here are performance numbers(execution time in seconds):
1.1 Baseline 1.5 1.5 + Decimal
q63 27 26 38
q98 18 26 24

As you can see q63 is showing regression compared to similar query. I am attaching the both version of queries and affected schemas. When adding the windows function back this is the only query seem to be slower than 1.1 in 1.5.
I have attached the both version of schema and queries.