Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2869

KafkaSpout discards all pending records when adjusting the consumer position after a commit

    Details

      Description

      As part of the STORM-2666 fix the spout clears out waitingToEmit when the consumer position falls behind the committed offset during a commit. We only need to do it for the affected partition, and then only for the records that are behind the committed offset.

      Also the validation check in emitOrRetryTuple is slightly too permissive, it should check whether the current record is behind the committed offset, not whether the consumer position is behind the committed offset.

        Attachments

          Activity

            People

            • Assignee:
              Srdo Stig Rohde Døssing
              Reporter:
              Srdo Stig Rohde Døssing
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 20m
                1h 20m