Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-15144

MM2 Checkpoint downstreamOffset stuck to 1

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Bug
    • None
    • None
    • mirrormaker
    • None

    Description

      Steps to reproduce :

      1.Start the source cluster

      2.Start the target cluster

      3.Start connect-mirror-maker.sh using a config like the attached

      4.Create a topic in source cluster

      5.produce a few messages

      6.consume them all with autocommit enabled

       

      7. then dump the Checkpoint topic content e.g.

      % bin/kafka-console-consumer.sh --bootstrap-server localhost:9192 --topic source.checkpoints.internal --from-beginning --formatter org.apache.kafka.connect.mirror.formatters.CheckpointFormatter

      Checkpoint{consumerGroupId=edogroup, topicPartition=source.vf-mirroring-test-edo-0, upstreamOffset=3, downstreamOffset=1, metadata=}

       

      the downstreamOffset remains at 1, while, in a fresh cluster pair like with the source topic created while MM2 is running,
      I'd expect the downstreamOffset to match the upstreamOffset.

      Note that dumping the offset sync topic, shows matching initial offsets

      % bin/kafka-console-consumer.sh --bootstrap-server localhost:9192 --topic mm2-offset-syncs.source.internal --from-beginning --formatter org.apache.kafka.connect.mirror.formatters.OffsetSyncFormatter

      OffsetSync{topicPartition=vf-mirroring-test-edo-0, upstreamOffset=0, downstreamOffset=0}
       
       
       

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ecomar Edoardo Comar
            ecomar Edoardo Comar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment