Major Description
After upgrading to Samza 1.2, the samza metrics high-watermark and messages-behind-high-watermark stopped updating. With a bit of digging, this was traced to our use of dot in our kafka topic names.
It looks like kafka refactored their metrics names significantly, and moved the topic name out of the metric name and into tags. When doing this, they added a replacement of '.' -> '_'. When Samza refactored to match this, the same replacement was not made, so the lookup of the "records-lag" metric in KafkaConsumerProxy fails to find a match.
The commit on kafka is here:
https://github.com/apache/kafka/commit/5d81639907869ce7355c40d2bac176a655e52074#diff-b45245913eaae46aa847d2615d62cde0R1331
The related commit on Samza is here:
https://github.com/apache/samza/commit/01c06055057e8c260e73bf3f7db7b9eb53af3745#diff-b56db493dfdd2fd2d471f6df342fe661R400
The fix is a simple replacement added to the line above. Pull request coming shortly.
Attachments
Issue Links
- links to
