Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2184

New Kafka spout will fail to commit offsets on shutdown if there are offsets to commit

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0, 1.1.0, 1.0.3
    • Component/s: storm-kafka-client
    • Labels:
      None

      Description

      As far as I know spout methods are not called concurrently by Storm. When the new Kafka spout closes or deactivates, the closing thread calls KafkaConsumer.wakeup. If there are uncommitted offsets and the spout is in manual commit mode, it then tries to commit offsets. Since there are no other threads concurrently interacting with the spout, the call to KafkaConsumer.commitSync will end up throwing WakeupException. This is because the behavior of wakeup is to cause the next blocking call to throw WakeupException if there are no other threads blocking on the consumer when wakeup is called.

      The call to wakeup should be removed, since it does nothing if there are no offsets to commit, and causes an uncaught exception if there are.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Srdo Stig Rohde Døssing
                Reporter:
                Srdo Stig Rohde Døssing
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m