1. The following code seems to be used to enable/disable the header. Is it better to control that in config instead using debug logging (so that it's not mixed with other debug logging)? Also, the header info is not complete, missing the first few fields. The header is probably useful for stats printed out periodically in each thread. So it should be printed out early, if enabled.
logger.debug("message size, batch size, total data sent in MB, MB/sec, total data sent in nMsg, nMsg/sec")
2. Whether avgPerf is specified or not, the user is probably always interested in the aggregated numbers across all threads. How about we always print it out and have a config option "showDetails" to enable/disable periodic reporting in each thread. Ditto in other perf tools.
3. ProducerThread has multiple bugs:
3.1. Variable-sized messages are not picked up in Async mode
3.2. In Sync mode, messageSet needs to be reset for each batch, if messages are of variable size (seems to be an existing bug)
4. It's better not to duplicate the following code. Defining it once in a static method seems better.
println(("%s, %d, %d, %d, %d, %.2f, %.4f, %d, %.4f").format(formattedReportTime, config.compressionCodec.codec,
threadId, config.messageSize, config.batchSize, (bytesSent*1.0)/(1024 * 1024), mbPerSec, nSends, numMessagesPerSec))