[KAFKA-2756] Replication Broken between Kafka 0.8.2.1 and 0.9 - NetworkClient.java uses wrong protocol version

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.9.0.0
• Fix Version/s: 0.9.0.0
• Assignee: Matthew Bruce
• Reporter: Matthew Bruce
• Created: 05/Nov/15 20:50
• Updated: 05/Nov/15 23:46
• Resolved: 05/Nov/15 23:46

Description

During a rolling upgrade from 0.8.2.1 to 0.9.0.0, replication between 0.9.0.0 and 0.8.2.1 fails due to
org.apache.kafka.clients.networkClient:handleCompletedReceives always using the latest version of the API Key available instead of the one specified by inter.broker.protocol.version.

This line should not use ProtoUtils.currentResponseSchema and instead call ProtoUtils.ResponseSchema and specify a version explicitly:

Struct body = (Struct) ProtoUtils.currentResponseSchema(apiKey).read(receive.payload());

This results in WARN messages like the following in the server.log file as the responses are decoded with the wrong Schema:

[2015-11-05 19:13:10,309] WARN [ReplicaFetcherThread-0-182050600], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@6cc18858. Possible cause: org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'responses': Error reading field 'topic': java.nio.BufferUnderflowException (kafka.server.ReplicaFetcherThread)

[2015-11-03 16:55:15,178] WARN [ReplicaFetcherThread-1-182050600], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@224388b2. Possible cause: org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'responses': Error reading field 'partition_responses': Error reading field 'record_set': java.lang.IllegalArgumentException (kafka.server.ReplicaFetcherThread)

Issue Links

• relates to: KAFKA-2750 Sender.java: handleProduceResponse does not check protocol version