[IMPALA-10504] Add tracing for remote block reads

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Epic Color: ghx-label-3
• Assignee: Kurt Deschler
• Reporter: Kurt Deschler
• Created: 12/Feb/21 17:02
• Updated: 31/Mar/21 04:38
• Resolved: 31/Mar/21 04:38

Description

While chasing performance issues, there were a large number of remote block read messages in the logs. Need tracing to track down the source of these. 

    Errors: Read 3.07 GB of data across network that was expected to be local.
Block locality metadata for table 'tpcds_600_parquet.store_sales' may be stale.
This only affects query performance and not result correctness.
One of the common causes for this warning is HDFS rebalancer moving some of the file's blocks.
If the issue persists, consider running "INVALIDATE METADATA `tpcds_600_parquet`.`store_sales`"