Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4362

Consumer can fail after reassignment of the offsets topic partition

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.1, 0.10.1.0
    • Fix Version/s: 0.10.1.1
    • Component/s: None
    • Labels:
      None

      Description

      When a consumer offsets topic partition reassignment completes, an offset commit shows this:

      java.lang.IllegalArgumentException: Message format version for partition 100 not found
          at kafka.coordinator.GroupMetadataManager$$anonfun$14.apply(GroupMetadataManager.scala:633) ~[kafka_2.10.jar:?]
          at kafka.coordinator.GroupMetadataManager$$anonfun$14.apply(GroupMetadataManager.scala:633) ~[kafka_2.10.jar:?]
          at scala.Option.getOrElse(Option.scala:120) ~[scala-library-2.10.4.jar:?]
          at kafka.coordinator.GroupMetadataManager.kafka$coordinator$GroupMetadataManager$$getMessageFormatVersionAndTimestamp(GroupMetadataManager.scala:632) ~[kafka_2.10.jar:?]
          at 
      ...
      

      The issue is that the replica has been deleted so the GroupMetadataManager.getMessageFormatVersionAndTimestamp throws this exception instead which propagates as an unknown error.

      Unfortunately consumers don't respond to this and will fail their offset commits.

      One workaround in the above situation is to bounce the cluster - the consumer will be forced to rediscover the group coordinator.

      (Incidentally, the message incorrectly prints the number of partitions instead of the actual partition.)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mgharat Mayuresh Gharat
                Reporter:
                jjkoshy Joel Koshy
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: