Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-3637

Looping topology structure can cause backpressure to deadlock

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0, 2.1.0
    • Fix Version/s: None
    • Component/s: storm-core
    • Labels:
      None

      Description

      When you have a topology structure with loops in it (BoltA and BoltB send tuples to each other), it can cause backpressure to deadlock.

      The scenario is that BoltA suddenly takes a long time to process a tuple (in our situation, it's doing a database operation). This causes the task input queue to fill up, setting the backpressure flag.

      BoltB, which is sending a tuple to BoltA, then cannot send, and the tuple is held in the emit queue. This blocks any tuples behind it, and also stops BoltB from executing. This means the input queue to BoltB will build up, until that backpressure flag is also set - and then when BoltA next wants to send a tuple to BoltB, it will irrevocably deadlock.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              thecoop1984 Simon Cooper

              Dates

              • Created:
                Updated:

                Issue deployment