[DERBY-6096] OutOfMemoryError with Clob or Blob hash join: DataTypeDescriptor.estimatedMemoryUsage() has no case for BLOB or CLOB so would underestimate memory usage for those types at zero

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.2.1, 10.7.1.1, 10.8.3.0, 10.9.1.0, 10.10.1.1
• Fix Version/s: 10.11.1.1
• Component/s: SQL
• Labels: derby_backport_reject_10_10
• Assignee: Katherine Marsden
• Reporter: Katherine Marsden
• Created: 04/Mar/13 14:13
• Updated: 09/Sep/16 09:53
• Resolved: 13/Jun/13 22:43

Description

In discussion on derby-dev regarding how much memory is used for hash joins, Knut noted:

I haven't verified, but I think HashJoinStrategy uses
DataTypeDescriptor.estimatedMemoryUsage() to estimate how much memory
the hash table will consume. That method has no case for BLOB or CLOB,
so it looks as if it will return zero for LOB columns. If that's so, it
will definitely overestimate how many rows fits in maxMemoryPerTable
kilobytes if the rows contain LOBs.

DataTypeDescriptor.estimatedMemoryUsage() should be updated to include BLOB and CLOB and we should try verify if this theory is correct with a reproduction.

Issue Links

• is related to: DERBY-4620 Query optimizer causes OOM error on a complex query