[ARROW-6803] [Rust] [DataFusion] Aggregate queries are slower with new physical query plan

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.16.0
• Fix Version/s: 0.16.0
• Component/s: Rust, Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/23137
• Assignee: Andy Grove
• Reporter: Andy Grove
• Created: 07/Oct/19 11:39
• Updated: 11/Jan/23 07:49
• Resolved: 08/Oct/19 20:11

Description

executing direct from logical plan:

 aggregate_query_no_group_by                                                                             
                        time:   [13.096 us 13.187 us 13.294 us]
                        change: [-88.712% -88.554% -88.398%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mildaggregate_query_group_by                                                                             
                        time:   [44.153 us 44.816 us 45.541 us]
                        change: [-77.984% -77.485% -77.009%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mildaggregate_query_group_by_with_filter                                                                            
                        time:   [75.383 us 76.076 us 76.817 us]
                        change: [-72.345% -71.811% -71.097%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)

executing from physical plan:

aggregate_query_no_group_by                                                                            
                        time:   [112.13 us 113.63 us 115.26 us]
                        change: [-3.8005% -2.0342% -0.3584%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mildaggregate_query_group_by                                                                            
                        time:   [195.12 us 198.63 us 202.39 us]
                        change: [-1.3814% +1.0612% +3.5732%] (p = 0.40 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severeaggregate_query_group_by_with_filter                                                                            
                        time:   [270.69 us 272.18 us 273.63 us]
                        change: [-2.1583% -0.4877% +1.0161%] (p = 0.56 > 0.05)
                        No change in performance detected.
 

Issue Links

• links to: GitHub Pull Request #5590