[BEAM-2551] KafkaIO reader blocks indefinitely if servers are not reachable

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1.0
• Component/s: io-java-kafka
• Assignee: Raghu Angadi
• Reporter: Raghu Angadi
• Created: 04/Jul/17 07:19
• Updated: 16/May/20 13:53
• Resolved: 10/Jul/17 20:01

Description

If the KafaIO source reader on the worker can't reach the server, Kafka consumer blocks forever inside UnboundedReader.start(). Users have no indication of the error. It is better if start() fails with an error.

It is easy to reproduce in Kafka. I reported it on Kafka users list here : https://lists.apache.org/thread.html/98cebefacbd65b0d6c6817fe0b5197e26bc60252e72d05fced91e628@%3Cusers.kafka.apache.org%3E

It blocks inside Kafka client. Fortunately it can be unblocked with KafkaConsumer.wakeup(). We could run initialization in another thread and cancel it if takes longer than a minute.

Issue Links

• links to: GitHub Pull Request #3492
• links to: GitHub Pull Request #3513