[KAFKA-509] server should shut down on encountering invalid highwatermark file

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.8.0
• Labels: bugs
• Assignee: Yang Ye
• Reporter: Swapnil Ghike
• Created: 12/Sep/12 23:58
• Updated: 02/Oct/12 15:48
• Resolved: 02/Oct/12 15:48

Description

1. Somehow I managed to produce the following .highwatermark file (most probably while playing with kafka-create-topic.sh) -
0
7
abra.kabra 0 0
0 0 0
abrakabra 0 0
\0 0 0
king. 0 0
abra..kabra 0 0
... 0 0

Perhaps the first two lines are not valid. But I am not able to reproduce this issue today.

2. Since the topic names can contain a whitespace, perhaps the delimiter should change from a space char to / which is not allowed anymore in topic names.

3. With this .highwatermark file, the kafka server produces following -
[2012-09-12 14:54:49,456] INFO Replica Manager on Broker 0: Becoming Leader for topic [abra.kabra] partition [0] (kafka.server.ReplicaManager)
[2012-09-12 14:54:49,456] INFO [ReplicaFetcherManager on broker 0, ], removing fetcher on topic abra.kabra, partition 0 (kafka.server.ReplicaFetcherManager)
[2012-09-12 14:54:49,457] ERROR Replica Manager on Broker 0: Error processing leaderAndISR request LeaderAndIsrRequest(1,,true,1000,Map((...,0) ->

{ "ISR": "0","leader": "0","leaderEpoch": "0" }

, (\0,0) ->

{ "ISR": "0","leader": "0","leaderEpoch": "0" }

, (abrakabra,0) ->

{ "ISR": "0","leader": "0","leaderEpoch": "0" }

, (0,0) ->

{ "ISR": "0","leader": "0","leaderEpoch": "0" }

, (abra..kabra,0) ->

{ "ISR": "0","leader": "0","leaderEpoch": "0" }

, (king.,0) ->

{ "ISR": "0","leader": "0","leaderEpoch": "0" }

, (abra.kabra,0) ->

{ "ISR": "0","leader": "0","leaderEpoch": "0" }

)) (kafka.server.ReplicaManager)
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1937)
at kafka.server.HighwaterMarkCheckpoint$$anonfun$1.apply(HighwaterMarkCheckpoint.scala:103)
at kafka.server.HighwaterMarkCheckpoint$$anonfun$1.apply(HighwaterMarkCheckpoint.scala:96)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at scala.collection.immutable.Range$ByOne$class.foreach(Range.scala:282)
at scala.collection.immutable.Range$$anon$2.foreach(Range.scala:265)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
at scala.collection.immutable.Range.map(Range.scala:39)
at kafka.server.HighwaterMarkCheckpoint.read(HighwaterMarkCheckpoint.scala:96)
... (more junk) ..

And then it goes in a while(1) loop to print the following -
[2012-09-12 14:54:50,458] ERROR Replica Manager on Broker 0: Highwatermark for topic abra.kabra partition 0 doesn't exist during checkpointing (kafka.server.ReplicaManager)
[2012-09-12 14:54:50,459] ERROR Replica Manager on Broker 0: Highwatermark for topic ... partition 0 doesn't exist during checkpointing (kafka.server.ReplicaManager)
[2012-09-12 14:54:50,459] ERROR Replica Manager on Broker 0: Highwatermark for topic \0 partition 0 doesn't exist during checkpointing (kafka.server.ReplicaManager)
[2012-09-12 14:54:50,459] ERROR Replica Manager on Broker 0: Highwatermark for topic 0 partition 0 doesn't exist during checkpointing (kafka.server.ReplicaManager)
[2012-09-12 14:54:50,459] ERROR Replica Manager on Broker 0: Highwatermark for topic king. partition 0 doesn't exist during checkpointing (kafka.server.ReplicaManager)
[2012-09-12 14:54:50,460] ERROR Replica Manager on Broker 0: Highwatermark for topic abra..kabra partition 0 doesn't exist during checkpointing (kafka.server.ReplicaManager)
[2012-09-12 14:54:50,460] ERROR Replica Manager on Broker 0: Highwatermark for topic abrakabra partition 0 doesn't exist during checkpointing (kafka.server.ReplicaManager)

The server should shut down on encountering the error.