There are a few deficiencies here:
- We don't track the peak number of scanner threads. Consumers of the profile often confuse NumScannerThreadsStarted with the peak.
- Kudu scans are missing some metrics, e.g. AverageScannerThreadConcurrency. We should make sure that Kudu and HDFS are consistent.
We should clean this up, and maybe refactor the code so that less logic is duplicated