[KAFKA-4362] Consumer can fail after reassignment of the offsets topic partition

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0.1, 0.10.1.0
• Fix Version/s: 0.10.1.1
• Assignee: Mayuresh Gharat
• Reporter: Joel Jacob Koshy
• Created: 01/Nov/16 00:10
• Updated: 05/Jun/17 21:46
• Resolved: 23/Nov/16 08:27

Description

When a consumer offsets topic partition reassignment completes, an offset commit shows this:

java.lang.IllegalArgumentException: Message format version for partition 100 not found
    at kafka.coordinator.GroupMetadataManager$$anonfun$14.apply(GroupMetadataManager.scala:633) ~[kafka_2.10.jar:?]
    at kafka.coordinator.GroupMetadataManager$$anonfun$14.apply(GroupMetadataManager.scala:633) ~[kafka_2.10.jar:?]
    at scala.Option.getOrElse(Option.scala:120) ~[scala-library-2.10.4.jar:?]
    at kafka.coordinator.GroupMetadataManager.kafka$coordinator$GroupMetadataManager$$getMessageFormatVersionAndTimestamp(GroupMetadataManager.scala:632) ~[kafka_2.10.jar:?]
    at 
...

The issue is that the replica has been deleted so the GroupMetadataManager.getMessageFormatVersionAndTimestamp throws this exception instead which propagates as an unknown error.

Unfortunately consumers don't respond to this and will fail their offset commits.

One workaround in the above situation is to bounce the cluster - the consumer will be forced to rediscover the group coordinator.

(Incidentally, the message incorrectly prints the number of partitions instead of the actual partition.)

Issue Links

• relates to: KAFKA-5381 ERROR Uncaught exception in scheduled task 'delete-expired-consumer-offsets' (kafka.utils.KafkaScheduler)
• links to: GitHub Pull Request #2116