[DERBY-3732] SQL Length function materializes BLOB into memory

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 10.3.3.0, 10.4.1.3, 10.5.1.1
• Fix Version/s: 10.3.3.1, 10.4.2.0, 10.5.1.1
• Component/s: SQL
• Issue & fix info: High Value Fix
• Assignee: Katherine Marsden
• Reporter: Katherine Marsden
• Created: 19/Jun/08 23:18
• Updated: 30/Jun/09 16:02
• Resolved: 03/Jul/08 14:47

Description

Currently the SQL length function materializes the entire lob into memory. In SQLBinary.getLength() we have
public final int getLength() throws StandardException
{
if (stream != null)

{ if (streamValueLength != -1) return streamValueLength; }

return (getBytes() == null) ? 0 : getBytes().length;
}
Which actually is doubly bad because we call getBytes twice and materialize it twice.
It would be good to read the length from the stream if available and otherwise stream the value to get the length, rather than materializing it into memory.

To reproduce, run the attached repro.
java -Xmx16M LengthLargeLob

It gives an out of memory exception
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.derby.iapi.types.SQLBinary.readFromStream(SQLBinary.java:415)
at org.apache.derby.iapi.types.SQLBinary.readExternal(SQLBinary.java:318)
at org.apache.derby.iapi.types.SQLBinary.getValue(SQLBinary.java:220)
at org.apache.derby.iapi.types.SQLBinary.getBytes(SQLBinary.java:210)
at org.apache.derby.iapi.types.SQLBinary.getLength(SQLBinary.java:250)
at org.apache.derby.impl.sql.execute.BaseActivation.getDB2Length(BaseActivation.java:1684)
at org.apache.derby.exe.acf81e0010x011axa317x5db8x0000003d9dc81.e1(Unknown Source)
at org.apache.derby.impl.services.reflect.DirectCall.invoke(ReflectGeneratedClass.java:141)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(ProjectRestrictResultSet.java:497)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:291)
at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
... 2 more
[

Issue Links

• relates to: DERBY-3741 SQL LENGTH function materializes CLOB into memory