[IMPALA-12544] Replace scan progress with query progress as progress reporting for the shell

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.3.0
• Fix Version/s: Impala 4.4.0
• Component/s: Clients
• Epic Color: ghx-label-8
• Assignee: Zihao Ye
• Reporter: Zihao Ye
• Created: 06/Nov/23 10:01
• Updated: 08/Apr/24 12:22
• Resolved: 10/Nov/23 16:18

Description

After IMPALA-12048 is resolved, we can see the query progress of a query on the /queries page. Unlike scan progress, it uses the completion count of query fragment instances to calculate progress, providing a more accurate reflection of query execution progress, especially for computation-intensive queries (such as TPC-DS query 78). Perhaps we should also use query progress instead of scan progress in the dynamic progress reporting for the impala shell, to provide more accurate query completion progress reports and avoid cases where the scan is complete but the query still needs a long time to finish computation even though the progress is reported as 100%.

Issue Links

• causes: IMPALA-12978 IMPALA-12544 made impala-shell incompatible with old impala servers