Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-3320

Executors should start when all worker connections are ready

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0
    • Component/s: None

      Description

      We conflate "being activated" with "all workers are ready" in WorkerState, by making isWorkerActivated a part of isTopologyActivated.

      The issue with this is that isTopologyActivated is used to communicate activation/deactivation to the executors, and is updated on a timer (default only every 10 seconds). isWorkerActivated is really meant to be a one-way switch, which lets us delay executor initialization until all other workers in the topology are also started.

      Since we mix the two up, if a worker is started in the topology and all other connections aren't ready immediately (e.g. as happens every time you deploy a topology, some workers will boot faster than others), the worker may have to wait up to 10 seconds to start.

      We should make sure the wait for isWorkerActivated happens via CountDownLatch instead, so the executor will start as soon as the connections are ready.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                srdo Stig Rohde Døssing
                Reporter:
                srdo Stig Rohde Døssing
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m