Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2678

Improve performance of LoadAwareShuffleGrouping

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: storm-client
    • Labels:
      None

      Description

      While reviewing STORM-2306, I found a room to improve LoadAwareShuffleGrouping, basically taking similar approach with ShuffleGrouping.

      The idea is constructing virtual ring (array) with proper length which tasks are randomly distributed to the ring respecting their loads, and just do sequential access like we do in ShuffleGrouping. Maybe slower to reflect load information due to construct virtual ring but no need to call randInt and additional loop for each tuple emission.

      We may also want to reduce call of System.currentTimeMillis(), and if we get rid of that, it should be better.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kabhwan Jungtaek Lim
                Reporter:
                kabhwan Jungtaek Lim
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 50m
                  3h 50m