Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-778

KafkaCheckpointMigration should not start/stop the same instance of CoordinatorStreamSystemConsumer multiple times

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:
      None

      Description

      The current implementation of KafkaSystemConsumer only allows start()/stop() once in the life cycle of the same SystemConsumer instance. Unfortunately, KafkaCheckpointMigration code start()/stop() the same instance of KafkaSystemConsumer multiple times, which results in an infinite loop in CoordinatorStreamSystemConsumer.start() in JobRunner.

      The quick fix is to avoid the sequence of SystemConsumer.start()>SystemConsumer.stop()>SystemConsumer.start() in the KafkaCheckpointMigration code.

      In long-term, we may also want to evaluate the use cases where we support start/stop multiple times in the life cycle of a single instance of SystemConsumer.

        Attachments

        1. SAMZA-778-0.patch
          0.8 kB
          Yi Pan

          Activity

            People

            • Assignee:
              nickpan47 Yi Pan
              Reporter:
              nickpan47 Yi Pan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: