Resolution: Cannot Reproduce
Affects Version/s: 5.6.0, 5.7.0, 5.8.0, 5.9.0
Fix Version/s: None
Client library 5.5.1
We experience this problem in combination with 5.5.1 client and the wireformat.tightEncodingEnabled=false+ failover:
1. start standard broker
2. start Client (with e.g. a MessageListener) with failover protocol: e.g. failover:(tcp://18.104.22.168:61616?wireformat.tightEncodingEnabled=false)
3. wait around 30sec (default for inactivity check)
The client closes the connection and re-tries to the broker which in turn throws the following exception:
The problem here is that the BaseDataStreamMarshaller reads an int from the buffer and re-uses it immediately to allocate a byte array:
In our case the dadaIn.readInt() read an int number of 785.477.224 which triggers the broker to allocate blindly this amount of mem.
We do not know yet what triggers the wrong byte sequence from the client, but on the brokers side, there should be a protection against this case.