[IMPALA-6977] Report CPU usage (user/sys) per fragment

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Labels: observability resource-management
• Epic Link: Improve Impala Profile User Experience
• Target Version: Product Backlog
• Epic Color: ghx-label-3
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 04/May/18 21:48
• Updated: 18/Oct/18 21:52

Description

The exec summary only shows wallclock time, because that is the only time reported per operator.

Operator       #Hosts  Avg Time  Max Time  #Rows  Est. #Rows   Peak Mem  Est. Peak Mem  Detail                 
---------------------------------------------------------------------------------------------------------------
03:AGGREGATE        1   1.782ms   1.782ms      1           1   28.16 KB       10.00 MB  FINALIZE               
02:EXCHANGE         1  90.073us  90.073us      3           1  104.00 KB              0  UNPARTITIONED          
01:AGGREGATE        3   1s560ms   1s657ms      3           1    1.35 MB       10.00 MB                         
00:SCAN HDFS        3   1s567ms   1s815ms  6.00M       6.00M   59.11 MB      144.00 MB  tpch_seq_gzip.lineitem 

This is fairly useful for understanding query bottlenecks but is misleading whn it comes to determining where CPU was consumed. We have accurate User and Sys time for fragment instances so we could have a CPU usage summary based on profile information

Fragment       #Hosts  Avg UserTime  Max UserTime  Avg SysTime   Max SysTime
------------------------------------------------------------------------------
F01            1       12.000ms      .......       ..........
F00            3       7s245ms       7s624ms       .......       ..........