[KAFKA-4102] Is kafka mirror maker in 0.10.X don't support mirror kafka 0.8.X?

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Assignee: Unassigned
• Reporter: yunfan
• Created: 30/Aug/16 03:36
• Updated: 03/Jan/17 23:08
• Resolved: 03/Jan/17 23:08

Description

I want this feature (https://cwiki.apache.org/confluence/display/KAFKA/KIP-3+-+Mirror+Maker+Enhancement) in kafka new version.
But it seems can't support kafka 0.8.X ?

Cmd: ./kafka_2.11-0.10.0.1/bin/kafka-mirror-maker.sh --producer.config svc_run/mirror_kafka_write_online2streaming/producer.properties --consumer.config svc_run/mirror_kafka_write_online2streaming/consumer.properties --whitelist sandbox

Producer config:
queue.buffering.max.messages=16384
bootstrap.servers=XXX
send.buffer.bytes=131072
message.send.max.retries=1048576

consumer config:
zookeeper.connect=XXXX
group.id=test_mirror
consumer.timeout.ms=-1
zookeeper.connection.timeout.ms=60000
zookeeper.session.timeout.ms=60000
socket.receive.buffer.bytes=-1
auto.commit.interval.ms=5000
auto.commit.enable=false

But I got failures like follows:

FetchRequest@33437902 (kafka.consumer.ConsumerFetcherThread)
java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:275)
at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.Range.foreach(Range.scala:160)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.immutable.Range.foreach(Range.scala:160)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)