Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2422

Serialized Trident topology size does not grow linerarly

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: trident
    • Labels:
      None

      Description

      Many of the Trident bolts contain a serialized version of the entire trident graph. This means that as the size of the graph grows, and the number of bolts grows with it, that the serialized topology grows quadraticly.

      The bolt only uses a small portion of that graph so we can filter out most of that.

      As an extreme example a topology with 1000 Bolts (4000 nodes in the graph) can grow to be over 1GB in size. If we trim out the parts of the graph that are not needed the serialized topology is < 40MB in size.

      This is a backwards compatible change, because we are only stripping out data that is never touched by the bolt anyways.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                revans2 Robert Joseph Evans
                Reporter:
                revans2 Robert Joseph Evans
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m