Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2087

Storm-kafka-client: Failed tuples are not always replayed

    Details

      Description

      I am working with kafka 10 and the storm-kafka-client from master. It appears that tuples are not always being replayed when they are failed.

      With a topology that randomly fails tuples a small percentage of the time I found that the committed kafka offset would get stuck and eventually processing would stop even though the committed offset was no where near the end of the topic.

      I have also replicated the issue in unit tests with this PR:
      https://github.com/apache/storm/pull/1679

      It seems that increasing the number of times I call nextTuple for the in order case will make it work, but it doesn't seem to help the case where tuples are failed out of order from which they were emitted.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jfenc91 Jeff Fenchel
                Reporter:
                jfenc91 Jeff Fenchel
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 11.5h
                  11.5h