Currently I see this every commit interval:
We have ~10 tasks in our topology, 4 topics, and 32 partitions per topic.
This means every commit interval we log a few hundred lines of the above
which is an order of magnitude chattier than anything else in the log
during normal operations.
To improve visibility of important messages, we should reduce the chattiness of normal commits and highlight abnormal commits. An example proposal:
existing message is fine at TRACE level for diagnostics
TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31
normal fast case, wrap them all up into one summary line
INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms
some kind of threshold / messaging in case it doesn't complete quickly or logs an exception
ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in 100ms