Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
Description
Even with KAFKA-15942, the test PlaintextConsumerTest.testAutoCommitIntercept flakes on the the initial setup (before using interceptors, so interceptors are unrelated here, except for being used later in the test).
The problem is that we are seeking two topic partitions to offset 10 and 20, respectively, but when we commit, we seem to have lost one of the offsets, likely due to a race condition.
When I output `subscriptionState.allConsumed` repeatedly, I get this output:
allConsumed: {topic-0=OffsetAndMetadata{offset=100, leaderEpoch=0, metadata=''}, topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}}
seeking topic-0 to FetchPosition{offset=10, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[localhost:58298 (id: 0 rack: null)], epoch=0}}
seeking topic-1 to FetchPosition{offset=20, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[localhost:58301 (id: 1 rack: null)], epoch=0}}
allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}, topic-1=OffsetAndMetadata{offset=20, leaderEpoch=null, metadata=''}}
allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}}
allConsumed: {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}, topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}}
autocommit start {topic-0=OffsetAndMetadata{offset=10, leaderEpoch=null, metadata=''}, topic-1=OffsetAndMetadata{offset=0, leaderEpoch=null, metadata=''}}
So we after we seek to 10 / 20, we lose one of the offsets, maybe because we haven't reconciled the assignment yet. Later, we get the second topic partition assigned, but the offset is initialized to 0.
We should investigate whether this can be made more like the behavior in the original consumer.
Attachments
Issue Links
- links to