Recently I've debugged heap overgrowth during an exchange on configuration with large amount of caches.
I found several problems:
- GridDhtPartitionsExchangeFuture.msgs field is not cleared after exchange. Easy fix.
- GridDhtPartitionsSingleMessage.partCntrsBytes is serialized as object and takes large amount of heap.Could be fixed by implementing more efficient serialization.
- BinaryMemoryAllocatorChunk heap buffer is not shrinked during exchange due to large message size. WIth 128 threads in sys pool it may take up to 7G of heap on exchange process(I observer 52M message size). This could be fixed by reducing max exchange message size.