Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2104

New Kafka spout crashes if partitions are reassigned while tuples are in-flight

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0, 1.1.0
    • Fix Version/s: 2.0.0, 1.1.0, 1.0.4
    • Component/s: storm-kafka-client
    • Labels:
      None

      Description

      The new KafkaSpout may throw NPEs if partitions are reassigned while tuples are in-flight. The ack function assumes that the spout instance is always responsible for tuples it emitted, which isn't true if partitions were reassigned since the tuple was emitted. The fail function also assumes that failed tuples should be replayed, which is useless if the tuple is for a partition the spout isn't assigned, since it then can't commit the tuple if it succeeds. Both functions should check that the spout instance is responsible for the incoming tuple before scheduling it for retry or adding it to the acked list.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Srdo Stig Rohde Døssing
                Reporter:
                Srdo Stig Rohde Døssing
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 20m
                  4h 20m