Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10355

Handle accidental deletion of source-topics as exceptional failure

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.0
    • Component/s: streams
    • Labels:
      None

      Description

      Today when source topics are accidently deleted (either before the Streams app is started or while app is already running), the consumer group leader would notice from the metadata refresh that this topic is missing, and then will re-trigger the rebalance. During the rebalance the leader then would encode a `INCOMPLETE_SOURCE_TOPIC_METADATA` in the assignment to let all threads to shutdown upon getting the error.

      However, this shutdown process is a graceful shutdown and would only print an ERROR log entry "Received error code INCOMPLETE_SOURCE_TOPIC_METADATA - shutdown". That means users would not be programmatically notified when this happens. We should handle it not as a graceful shutdown but as a Shutdown a Kafka Streams Application error case which would trigger user's error handler with appropriate error reported.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cadonna Bruno Cadonna
                Reporter:
                guozhang Guozhang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: