It's hard to debug a hanging DDL/DML:
- The last log before coordinator sending execDdl RPC to catalogd is "Analysis and authorization finished."
- Catalogd just prints logs when it finishes the HMS operations and start reloading metadata incrementally.
It'd be helpful to add logs:
- On coordinator side, add logs when start/finish the execDdl RPCs.
- On catalogd side, add logs when start/finish executing an execDdl request. More loggings in specifit complex ddls will also be helpful.
We also need to add thread annotation in catalogd for the part executing HMS RPCs. We currently only have them in loading metadata. That's not enough.
CC Vihang Karajgaonkar who may already have a draft for this.