[IMPALA-521] Create/centralize a single wait timer for all external components in runtime profile

• Type: Task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 1.1
• Fix Version/s: Impala 1.2
• Assignee: Nong Li
• Reporter: Nong Li
• Created: 08/Aug/13 06:31
• Updated: 20/Dec/15 00:05
• Resolved: 21/Sep/13 01:38

Description

It would be useful to be easily to look at the runtime profile and figure out what is slow. A high level counter for each external component for the time Impala spent waiting would be very useful.

It might be most easy to diagnose to have this be CPU (i.e. thread time)

Examples would be:
Total wait time in HDFS/HBASE
Total wait time in network IO
Total wait time in client
Total wait time in hive metastore
Total CPU time on all threads for this query

Total CPU - Sum(External) is the time spent in Impala doing useful work.

This would very easily identify which part of the system was the bottleneck.