[IGNITE-5093] Better heap usage during exchange on large topologies and cache numbers/partitions.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.6
• Fix Version/s: 2.3
• Component/s: general
• Assignee: Alexey Scherbakov
• Reporter: Alexey Scherbakov
• Created: 26/Apr/17 14:23
• Updated: 28/Aug/17 16:37
• Resolved: 24/Jul/17 09:35

Description

I observed huge heap occupation on large grid installation including 136 nodes/1k caches.

Example from machine with 64g heap:

 num     #instances         #bytes  class name
----------------------------------------------
   1:       897287977    43069822896  java.util.HashMap$Node
   2:       9273162    14866180592  [Ljava.util.HashMap$Node;
   3:       201282292    4830775008    java.lang.Integer
   4:       6247215    983811096      [Ljava.lang.Object;
   5:       3383402    767741664      [C
   6:       12188    669411952       [B
   7:       9923859   635126976       java.util.HashMap
...

Further investigation had showed the heap is polluted during exchange process, which involves creating many hashmaps occupying large amounts of memory.

Proposal: use other datastructures to help keep heap usage low.