Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20194

KafkaSourceFetcherManager.commitOffsets() should handle the case when there is no split fetcher.

    XMLWordPrintableJSON

    Details

      Description

      The KafkaSourceFetcherManager.commitOffsets() may throw NPE when there is no split fetcher. Given that such cases should usually be rare and only happens when the source is shutting down or just started. We can create a new KafkaPartitionSplitReader to commit offset and close it immediately. The main thread maybe blocked for some time, but given that there is no event to process at that point, there should be no performance concerns.

        Attachments

          Issue Links

          blocks

          Sub-task - The sub-task of the issue FLINK-20114 Test Kafka Source based on the new Source API 

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. FLINK-20157 SourceCoordinatorProvider kills JobManager with IllegalStateException on job submission

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          links to

          Web Link GitHub Pull Request #14142

            Activity

              People

              • Assignee:
                becket_qin Jiangjie Qin
                Reporter:
                becket_qin Jiangjie Qin
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: