Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-11058

Possible OOM due to large discard queue in TcpDiscoverySpi

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8
    • Component/s: None
    • Labels:
      None

      Description

      Currently it's necessary to store every ensured (marked with TcpDiscoveryEnsureDelivery annotation) in pending message queue until it's discarded from coordinator for implementing guaranteed delivery, otherwise if subsequent nodes will fail while forwarding message the guarantee couldn't be fulfilled.

      On large topologies with active changes the queue may contain many very large messages causing heap usage bursts and possible OOM.

      Possible solution:

      1. off-load pending messages payload to off-heap or even on disk.
      2. store messages in serialized form for avoiding JVM Object overhead.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                antonovsergey93 Sergey Antonov
                Reporter:
                ascherbakov Alexei Scherbakov
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m