A topology that consumes from two different Kafka clusters: 0.10.1.1 and 0.10.2.1.
Spouts consuming from 0.10.2.1 have a low lag (and regularly commit offsets)
The Spout that consumes from 0.10.1.1 exhibits either:
1- Unknown lag
2- Lag that increments as the Spout reads messages from Kafka
In DEBUG, Offset manager logs: "topic-partition has NO offsets ready to be committed", despite continuing to consume messages.
Several configuration tweaks were tried, including setting maxRetries to 1, in case messages with a lower offset were being retried (logs didn't show it, though)
offsetCommitPeriodMs was also lowered to no avail.
The only configuration that works is to have ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG=true, but this is undesired since we lose processing guarantees.