Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5368

Let Kafka consumer show something when it fails to read one topic out of topic list

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0, 1.3.0
    • Component/s: Kafka Connector
    • Labels:
      None

      Description

      As a developer when reading data from many topics, I want Kafka consumer to show something if any topic is not available. The motivation is we read many topics as list at one time, and sometimes we fail to recognize that one or two topics' names have been changed or deprecated, and Flink Kafka connector doesn't show the error.

      My proposed change would be either to throw RuntimeException or to use LOG.error(topic + "doesn't have any partition") if partitionsForTopic is null at this function.
      https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.java#L208

      Any suggestion is welcome.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                HungChang Sendoh
                Reporter:
                HungChang Sendoh
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: