[IMPALA-12024] For CTAS, include time to create the table in the query profile timeline

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.3.0
• Component/s: Catalog
• Epic Link: Catalog Observability
• Epic Color: ghx-label-11
• Assignee: Quanlong Huang
• Reporter: Joe McDonnell
• Created: 23/Mar/23 17:37
• Updated: 12/Mar/24 01:41
• Resolved: 30/Aug/23 05:25

Description

If the metastore is slow, catalog operations to create / drop tables can take a long time. For a CTAS statement, we don't have a clear indication in the profile that this is happening. Instead, it shows up as a gap between planning finished and submitting for admission:

    Query Timeline: 12s696ms
       - Query submitted: 64.303us (64.303us)
       - Planning finished: 117.816ms (117.752ms)
       - Submit for admission: 12s360ms (12s243ms)
...

This may apply for other statement types.

To reproduce with CTAS on an Impala devenv:

1. Start Impala
2. In impala-shell, run "select * from functional.alltypes"
3. Find the metastore pid (jps -v | grep metastore)
4. sudo gdb attach ${metastore_pid}
5. In impala-shell, run "create table foo as select * from functional.alltypes"
.. This will hang because it can't create the table, wait a bit
6. Back in the gdb session, detach from metastore and let the CTAS finish