I managed to reproduce the error and get the stack trace, but this time it happened during the creation of an index. Larry reported to have seen the error during index creation.
It happens in the same code area as suggested by Knut Anders. I also observed the error once during compress in a different run, but a reboot of the database had overwritten the log before I could look at it.
I believe Knut's suggested fix will allow the vector to grow until it reaches Integer.MAX_VALUE, given there is enough heap memory to do so.
I have only tested the calculation itself, as each of my test runs with the repro takes many hours.
If we get this into the next release candidate, I can do some more test runs with it.
2009-03-30 01:23:21.340 GMT Thread[DRDAConnThread_8,5,main] (XID = 12059858), (SESSIONID = 421), (DATABASE = db), (DRDAID = NF000001.F2EB-4196790664386933061
), Failed Statement is: CREATE INDEX my_index ON my_table(my_col)
java.lang.IllegalArgumentException: Illegal Capacity: -18547753
at org.apache.derby.impl.store.access.sort.MergeSort.multiStageMerge(Unknown Source)
at org.apache.derby.impl.store.access.sort.MergeSort.openSortRowSource(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.openSortRowSource(Unknown Source)
at org.apache.derby.impl.sql.execute.CreateIndexConstantAction.loadSorter(Unknown Source)
at org.apache.derby.impl.sql.execute.CreateIndexConstantAction.executeConstantAction(Unknown Source)
at org.apache.derby.impl.sql.execute.MiscResultSet.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeUpdate(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLIMM(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Cleanup action completed