[PROTON-597] Improve the memory footprint of TrasportImpl

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: proton-0.7
• Fix Version/s: proton-0.8
• Component/s: proton-j
• Assignee: Robbie Gemmell
• Reporter: Marcel Meulemans
• Created: 06/Jun/14 14:38
• Updated: 07/Jan/16 11:37
• Resolved: 27/Jun/14 10:12

Description

The memory footprint of the TrasportImpl can be quite high when using large maxFrameSizes (for example 1MB). In the TrasportImpl of proton-j there are two classes TrasportOutputAdaptor and FrameParser (TrasportImpl has a member of both) that hold a ByteBuffer of maxFrameSize throughout their lifetime. In the case of the 1MB frame size that means that each transport has a fairly high memory footprint (2MB) considering that this memory is hardly ever used, i.e. only when there is a frame to transfer in/out and in most cases the frame will not be maxFrameSize. Because there is a Transport for every connection the memory usage can become quite high when dealing with many connections with large maxFrameSize, e.g. 2GB for only 1000 connections.

An proof of concept implementation for reducing the memory footprint by only allocating the buffers when needed can be found here:
https://github.com/marcelmeulemans/qpid-proton/

See also the short mailinglist disscusion here:
http://qpid.2158936.n2.nabble.com/High-memory-footprint-of-proton-j-in-ActiveMQ-td7608977.html