Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2087

Storm-kafka-client: Failed tuples are not always replayed

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      I am working with kafka 10 and the storm-kafka-client from master. It appears that tuples are not always being replayed when they are failed.

      With a topology that randomly fails tuples a small percentage of the time I found that the committed kafka offset would get stuck and eventually processing would stop even though the committed offset was no where near the end of the topic.

      I have also replicated the issue in unit tests with this PR:
      https://github.com/apache/storm/pull/1679

      It seems that increasing the number of times I call nextTuple for the in order case will make it work, but it doesn't seem to help the case where tuples are failed out of order from which they were emitted.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              jfenc91 Jeffrey Ryan Fenchel
              Reporter:
              jfenc91 Jeffrey Ryan Fenchel

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 11.5h
                11.5h

                  Issue deployment