[KAFKA-6909] Message Accumulation in case of no connection to Kafka

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: clients
• Assignee: Unassigned
• Reporter: Alexander Stepanov
• Created: 16/May/18 13:43
• Updated: 16/May/18 13:43

Description

Simple situation:

0) We have service A which works with Kafka as client

1) No connection to Kafka

2) Service A accumulates messages for sending to Kafka, but still no connection to Kafka

3) Then service A is crashed for some reason, for example, OOM or just server is dead

4) How to recover accumulated messages?

I think - it will be good to have some general code basement to persist service's outgoing messages to Kafka. For example, if service A is crashed, it can fetch messages from some local storage on startup and resend it to Kafka.