[DERBY-6309] Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while importing data

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 10.10.1.1
• Component/s: SQL
• Urgency: Normal
• Bug behavior facts: Crash
• Assignee: Unassigned
• Reporter: Manikandan Chellappan
• Created: 05/Aug/13 06:05
• Updated: 08/Jan/14 22:31

Description

While importing large files (> 500 MB) using SYSCS_UTIL.SYSCS_IMPORT_DATA procedure, derby runs out of heap space. The java process which is running the derby has 2GB of heap space. The following exception is logged in the derby.log.

ERROR 38000: The exception 'java.lang.OutOfMemoryError: Java heap space' was thrown while evaluating an expression.
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown Source)
at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown Source)
at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown Source)
at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
at org.apache.derby.impl.load.Import.performImport(Unknown Source)
at org.apache.derby.impl.load.Import.importData(Unknown Source)
at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
at oracle.retail.stores.iddi.IDDIDerbyWriter.importTable(IDDIDerbyWriter.java:233)
at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:488)
at oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.derby.impl.load.ImportReadData.readNextDelimitedRow(Unknown Source)
at org.apache.derby.impl.load.ImportReadData.readNextRow(Unknown Source)
at org.apache.derby.impl.load.ImportAbstract.next(Unknown Source)
at org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown Source)
at org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown Source)
at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown Source)
at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown Source)
at org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown Source)
at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
at org.apache.derby.impl.load.Import.performImport(Unknown Source)
at org.apache.derby.impl.load.Import.importData(Unknown Source)
at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown Source)
at org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown Source)
at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)

Issue Links

• relates to: DERBY-6111 OutOfMemoryError using CREATE INDEX on large tables