Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2348

Notifying BEM about kafkaConsumerProxy failure via kafkaConsumer to prevent BEM poll blocking in case of proxy failures

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 1.3
    • None
    • None

    Description

      The problem is that a container's startup blocks indefinitely during Changelog / Checkpoint / Coordinator stream read, 
      whenever the KafkaConsumerProxy fails for some reason (e.g., TopicAuth).

      This is because the SSPIterator keeps blocking on the blocking poll even when KafkaConsumerProxy stops due to exceptions thrown from KafkaConsumer.

      The fix is to allow KafkaConsumerProxy to report exceptions to KafkaSystemConsumer, which then relays it to BEM.
      BEM checks if there are any exceptions when doing a poll and re-throws.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rayman7718 Rayman
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m