Issue Details (XML | Word | Printable)

Key: DERBY-106
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Unassigned
Reporter: Gerald Khin
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Derby

HashJoinStrategy leads to java.lang.OutOfMemoryError

Created: 17/Dec/04 04:08 PM   Updated: 12/Jul/06 12:05 AM
Return to search
Component/s: SQL
Affects Version/s: None
Fix Version/s: 10.1.1.0

Time Tracking:
Not Specified

Resolution Date: 23/Mar/05 06:58 AM


 Description  « Hide
My application is running out of memory: I encounterd a java.lang.OutOfMemoryError. I used -Xmx256M. Unfortunatley, I cannot spend an arbitrary amount of JVM memory.

Then, I commented out the line in class OptimizerFactoryImpl which was adding the HashJoinStrategy to the set of Join strategies:

if (joinStrategySet == null)
{
// JoinStrategy[] jss = new JoinStrategy[2];
JoinStrategy[] jss = new JoinStrategy[1];
jss[0] = new NestedLoopJoinStrategy();
// jss[1] = new HashJoinStrategy();
joinStrategySet = jss;
}

And with these changes the OutOfMemoryError has gone away! And it works even with -Xmx128M!!!

So I guess that there is a major memory issue with this HashJoin strategy implementation.

If it turns out to be too complicated to make the memory consumption more predicatble or even bounded to some configurable limit, then I need at least as a workaround a way to turn off the HashJoin strategy completely: I did it by patching and building my own derby.jar, but if there would be an official solution with some kind of switch like a system property, it would be great!



 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Repository Revision Date User Message
ASF #157861 Thu Mar 17 02:35:44 UTC 2005 mikem committing change by Jack Klebanoff (klebanoff-derby@sbcglobal.net).

Fix for DERBY-106

This change fixes the BackingStoreHashTable class to actually spill
over to disk when necessary. Prior to this fix it was possible for
hash joins to fail with out of memory errors, when they should instead
have spilled to disk.
Files Changed
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/store/access/BackingStoreHashTableFromScan.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/compile/OptimizerImpl.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/HashTableResultSet.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/TestDiskHashtable.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/iapi/sql/compile/Optimizable.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/compile/FromBaseTable.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/ScrollInsensitiveResultSet.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/TestDiskHashtable.out
MODIFY /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/build.xml
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/compile/NestedLoopJoinStrategy.java
ADD /incubator/derby/code/trunk/java/engine/org/apache/derby/iapi/store/access/DiskHashtable.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/compile/Level2OptimizerImpl.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/iapi/sql/compile/Optimizer.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/compile/FromTable.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/SpillHash.out
MODIFY /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/derbylang.runall
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/iapi/store/access/BackingStoreHashtable.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/HashScanResultSet.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/iapi/sql/compile/JoinStrategy.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/sql/compile/HashJoinStrategy.java
MODIFY /incubator/derby/code/trunk/build.xml
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SpillHash.java