[DIRMINA-637] SSLEngine output buffer seems to be too small

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.1.1, 1.1.7
• Fix Version/s: 1.1.8
• Component/s: Filter
• Assignee: Emmanuel Lécharny
• Reporter: Dan Mihai Dumitriu
• Created: 17/Nov/08 20:54
• Updated: 05/Sep/14 09:37
• Resolved: 05/Sep/14 09:37

Description

the code below is in SSLHandler.java. it makes the assumption that the size of the output will never be larger than 2x the size of the input. that assumption appears to not hold up. It looks like this code has been fixed in trunk, but not in 1.1.7. we only see an error for VERY specific content, i.e. almost never.

public void encrypt(ByteBuffer src) throws SSLException {
if (!initialHandshakeComplete)

{ throw new IllegalStateException(); }

// The data buffer is (must be) empty, we can reuse the entire
// buffer.
outNetBuffer.clear();

// Loop until there is no more data in src
while (src.hasRemaining()) {

if (src.remaining() > ((outNetBuffer.capacity() - outNetBuffer
.position()) / 2)) {
// We have to expand outNetBuffer
// Note: there is no way to know the exact size required, but enrypted data
// shouln't need to be larger than twice the source data size?
outNetBuffer = SSLByteBufferPool.expandBuffer(outNetBuffer, src
.capacity() * 2);
if (SessionLog.isDebugEnabled(session))

{ SessionLog.debug(session, " expanded outNetBuffer:" + outNetBuffer); }

}

SSLEngineResult result = sslEngine.wrap(src, outNetBuffer);
if (SessionLog.isDebugEnabled(session))

{ SessionLog.debug(session, " Wrap res:" + result); }

if (result.getStatus() == SSLEngineResult.Status.OK) {
if (result.getHandshakeStatus() == SSLEngineResult.HandshakeStatus.NEED_TASK)

{ doTasks(); }

} else

{ throw new SSLException("SSLEngine error during encrypt: " + result.getStatus() + " src: " + src + "outNetBuffer: " + outNetBuffer); }

}

outNetBuffer.flip();
}