Uploaded image for project: 'Derby'
  1. Derby
  2. DERBY-3221

"java.sql.SQLException: The conglomerate (-5) requested does not exist." from Derby 10.3.1.4 embedded within Eclipse 3.3 and RAD 7.0

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 10.3.1.4, 10.3.2.1
    • Fix Version/s: 10.3.3.0, 10.4.1.3
    • Component/s: JDBC
    • Labels:
      None
    • Environment:
      Windows Vista Ubuntu Linux on IBM's VM (RAD 7.0)
    • Bug behavior facts:
      Regression

      Description

      We are getting an SQLException when several prepared statement deletes are done upon an existing database. As far as we can tell this exception should never occur unless (evil) things like deleting the database or editing files occurs. This is using the embedded driver within a plug-in inside RAD 7.0 (and Eclipse 3.3).

      I'm not sure what else to submit that might be helpful.

      java.sql.SQLException: The conglomerate (-5) requested does not exist.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
      at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at com.surelogic.sierra.jdbc.LazyPreparedStatementConnection$LazyPreparedStatement.invoke(Unknown Source)
      at $Proxy1.execute(Unknown Source)
      at com.surelogic.sierra.jdbc.finding.FindingManager.delete(Unknown Source)
      at com.surelogic.sierra.jdbc.finding.ClientFindingManager.updateLocalFindings(Unknown Source)
      at com.surelogic.sierra.jdbc.project.ClientProjectManager.synchronizeProject(Unknown Source)
      at com.surelogic.sierra.client.eclipse.jobs.SynchronizeJob.synchronize(Unknown Source)
      at com.surelogic.sierra.client.eclipse.jobs.SynchronizeJob.run(Unknown Source)
      at org.eclipse.core.internal.jobs.Worker.run(Unknown Source)
      Caused by: ERROR XSAI2: The conglomerate (-5) requested does not exist.
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source)
      at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source)
      at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source)
      at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source)
      at org.apache.derby.impl.sql.execute.IndexChanger.finish(Unknown Source)
      at org.apache.derby.impl.sql.execute.IndexSetChanger.finish(Unknown Source)
      at org.apache.derby.impl.sql.execute.RowChangerImpl.finish(Unknown Source)
      at org.apache.derby.impl.sql.execute.UpdateResultSet.open(Unknown Source)
      at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
      ... 14 more

      1. derby-3221.v3.diff
        5 kB
        Dyre Tjeldvoll
      2. derby-3221.v2.diff
        5 kB
        Dyre Tjeldvoll
      3. derby-3221.v1.diff
        2 kB
        Dyre Tjeldvoll
      4. derby-3221.prelim.diff
        0.5 kB
        Dyre Tjeldvoll
      5. SubShape.properties
        3 kB
        Dyre Tjeldvoll
      6. conlomerate.tar.gz
        8 kB
        James Alan Shepherd

        Issue Links

          Activity

          Hide
          army A B added a comment -

          dyre> my understanding is that there are no specific plans to make another 10.3 release

          For what it's worth, I think that in theory it should be safe to use a "personal" build off the 10.3 branch in a production environment. At least, that is one of the goals of a stable branch, so far as I understand it. Note the following line from the Apache Derby home page:

          http://db.apache.org/derby/dev/derby_source.html#Branches

          "Production use of Derby should use official Derby releases or snapshot builds off
          the branches (or self built jars off the branches with bug fixes)."

          The parenthetical at the end is what I'm referring to. So Tim, you could in theory build your own jars off the 10.3 branch and use that in production, if waiting for 10.4 is not an option...

          Unless of course the above-quoted page is wrong, in which case I hope someone a) corrects me rather quickly, and b) updates the page accordingly...

          Show
          army A B added a comment - dyre> my understanding is that there are no specific plans to make another 10.3 release For what it's worth, I think that in theory it should be safe to use a "personal" build off the 10.3 branch in a production environment. At least, that is one of the goals of a stable branch, so far as I understand it. Note the following line from the Apache Derby home page: http://db.apache.org/derby/dev/derby_source.html#Branches "Production use of Derby should use official Derby releases or snapshot builds off the branches (or self built jars off the branches with bug fixes)." The parenthetical at the end is what I'm referring to. So Tim, you could in theory build your own jars off the 10.3 branch and use that in production, if waiting for 10.4 is not an option... Unless of course the above-quoted page is wrong, in which case I hope someone a) corrects me rather quickly, and b) updates the page accordingly...
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Hi Tim,

          my understanding is that there are no specific plans to make another 10.3 release, (such a release would be numbered 10.3.3.x for some x which depends on the number of release candidates needed). Check out Kathey Marsden's reply to my related question about this on derby-dev:

          http://www.nabble.com/Confused-about-merging-and-fix-versions-to15092390.html

          As the release manager for 10.4 (which definitely will have this fix) I guess it is my job to convince you to switch to that Anyway, you can read about it here (please note that as this is open-source the dates on this page are goals, not definite promises):

          http://wiki.apache.org/db-derby/DerbyTenFourRelease

          Show
          dyret Dyre Tjeldvoll added a comment - Hi Tim, my understanding is that there are no specific plans to make another 10.3 release, (such a release would be numbered 10.3.3.x for some x which depends on the number of release candidates needed). Check out Kathey Marsden's reply to my related question about this on derby-dev: http://www.nabble.com/Confused-about-merging-and-fix-versions-to15092390.html As the release manager for 10.4 (which definitely will have this fix) I guess it is my job to convince you to switch to that Anyway, you can read about it here (please note that as this is open-source the dates on this page are goals, not definite promises): http://wiki.apache.org/db-derby/DerbyTenFourRelease
          Hide
          hallorant Tim Halloran added a comment -

          Any idea when 10.3.2.2 will be released? I couldn't find a download for it yet on the site. Best regards

          Show
          hallorant Tim Halloran added a comment - Any idea when 10.3.2.2 will be released? I couldn't find a download for it yet on the site. Best regards
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Merged to 10.3 with revision 615273

          Show
          dyret Dyre Tjeldvoll added a comment - Merged to 10.3 with revision 615273
          Hide
          dyret Dyre Tjeldvoll added a comment -

          I would think so. I can try to do that tomorrow. I probably need to practice my merging skills...

          Show
          dyret Dyre Tjeldvoll added a comment - I would think so. I can try to do that tomorrow. I probably need to practice my merging skills...
          Hide
          kmarsden Kathey Marsden added a comment -

          Would it make sense to port this to 10.3?

          Show
          kmarsden Kathey Marsden added a comment - Would it make sense to port this to 10.3?
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Committed revision 613116.

          Show
          dyret Dyre Tjeldvoll added a comment - Committed revision 613116.
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Attaching v3 to address Knut's latest comments. I was about to say that he just has to live with the white space diff since that's what Netbeans does, but then I softened up and edited the file in emacs...

          Show
          dyret Dyre Tjeldvoll added a comment - Attaching v3 to address Knut's latest comments. I was about to say that he just has to live with the white space diff since that's what Netbeans does, but then I softened up and edited the file in emacs...
          Hide
          knutanders Knut Anders Hatlen added a comment -

          Patch v2 looks good. Adding a getter and a setter for CID and making it private was a good move. +1 to commit.

          Two small issues that you may want to address:

          • The patch adds some whitespace in TemporaryRowHolderImpl.java right above makeIndexProperties()
          • It looks a bit odd to have an assert inside the mutator to ensure that it doesn't mutate the field. It also makes the method useless for others who may have a perfectly legitimate reason to mutate the field. Perhaps it's better to move the assert to the caller, since that's the context where the assert is meaningful.
          Show
          knutanders Knut Anders Hatlen added a comment - Patch v2 looks good. Adding a getter and a setter for CID and making it private was a good move. +1 to commit. Two small issues that you may want to address: The patch adds some whitespace in TemporaryRowHolderImpl.java right above makeIndexProperties() It looks a bit odd to have an assert inside the mutator to ensure that it doesn't mutate the field. It also makes the method useless for others who may have a perfectly legitimate reason to mutate the field. Perhaps it's better to move the assert to the caller, since that's the context where the assert is meaningful.
          Hide
          dyret Dyre Tjeldvoll added a comment -

          New patch (v2) (described in previous comments).

          Show
          dyret Dyre Tjeldvoll added a comment - New patch (v2) (described in previous comments).
          Hide
          dyret Dyre Tjeldvoll added a comment -

          I looked into the use of
          TemporaryRowHolderResultSet.reStartScan(long, long) some more, and
          it turns out that it is only used a couple of places in
          DependentResultSet. In all of those cases the two arguments are
          obtained from the very same TemporaryRowHolderImpl object
          which this TemporaryRowHolderResultSet refers to. So those
          parameters are unnecessary, and the temporary conglomerate id is
          never actually modified by reStartScan().

          I'm currently testing a new patch which does include a mutator for the CID
          variable, but which also has an ASSERT which verifies that the
          new and old values are identical.

          As suggested by Bryan, I defer a more thorough cleanup to a later patch.

          Show
          dyret Dyre Tjeldvoll added a comment - I looked into the use of TemporaryRowHolderResultSet.reStartScan(long, long) some more, and it turns out that it is only used a couple of places in DependentResultSet. In all of those cases the two arguments are obtained from the very same TemporaryRowHolderImpl object which this TemporaryRowHolderResultSet refers to. So those parameters are unnecessary, and the temporary conglomerate id is never actually modified by reStartScan(). I'm currently testing a new patch which does include a mutator for the CID variable, but which also has an ASSERT which verifies that the new and old values are identical. As suggested by Bryan, I defer a more thorough cleanup to a later patch.
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Thanks James,
          I took your DERBY-39 repro for a spin, but unfortunately I keep hitting DERBY-3310 and this time I don't seem to be able to work around it with a cast, so I need to manually disable the ASSERT, I think.

          I'm currently running tests for a new version of my patch, where I

          • revert back to testing the presence of a temp conglom with CID==0
          • make CID private
          • make clients (TemporaryRowHolderResultSet) use the existing accessor (getTemporaryConglomId())
          • always set CID to 0, when conglomCreated is set to false and the conglomerate is removed

          Right now I'm providing a mutator for CID that reStartScan can use, but I cannot see what purpose this really serves. Changing the CID behind the owning class' back like that seems like it has the potential to introduce both leaks and inconsistencies. I'm trying to flag any situation where a conglomerate could be leaked, or the holder left in an inconsistent state, and see if this actually happens in the tests.

          Show
          dyret Dyre Tjeldvoll added a comment - Thanks James, I took your DERBY-39 repro for a spin, but unfortunately I keep hitting DERBY-3310 and this time I don't seem to be able to work around it with a cast, so I need to manually disable the ASSERT, I think. I'm currently running tests for a new version of my patch, where I revert back to testing the presence of a temp conglom with CID==0 make CID private make clients (TemporaryRowHolderResultSet) use the existing accessor (getTemporaryConglomId()) always set CID to 0, when conglomCreated is set to false and the conglomerate is removed Right now I'm providing a mutator for CID that reStartScan can use, but I cannot see what purpose this really serves. Changing the CID behind the owning class' back like that seems like it has the potential to introduce both leaks and inconsistencies. I'm trying to flag any situation where a conglomerate could be leaked, or the holder left in an inconsistent state, and see if this actually happens in the tests.
          Hide
          northshorefiend James Alan Shepherd added a comment -

          FYI: DERBY-39 may be connected?

          I can't see how, but I can use almost the same test case as above to reproduce DERBY-39

          Show
          northshorefiend James Alan Shepherd added a comment - FYI: DERBY-39 may be connected? I can't see how, but I can use almost the same test case as above to reproduce DERBY-39
          Hide
          bryanpendleton Bryan Pendleton added a comment -

          > I really loathe such "cooperating classes" where correct behavior depends
          > on a shared state that's manipulated by both

          I think you are right to criticize this implementation. Perhaps a reasonable
          approach is to first make and verify the small fix, then later, separately,
          re-design this interaction as you suggest, with an explicit API for tracking
          whether or not the temporary row holder overflowed to an explicit conglomerate or not.

          Show
          bryanpendleton Bryan Pendleton added a comment - > I really loathe such "cooperating classes" where correct behavior depends > on a shared state that's manipulated by both I think you are right to criticize this implementation. Perhaps a reasonable approach is to first make and verify the small fix, then later, separately, re-design this interaction as you suggest, with an explicit API for tracking whether or not the temporary row holder overflowed to an explicit conglomerate or not.
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Sigh... you are probably right. I guess that the safest thing to do is to stick with the CID==0 test, and set it to 0 whenever the temporary conglomerate is removed. But I think that a more explicit and intuitive predicate would have made the code easier to read. Come to think of it, I really loathe such "cooperating classes" where correct behavior depends on a shared state that's manipulated by both... but in this case I guess a smaller, less intrusive, change is preferable.

          Wrt. reStartScan(): I would assume that when you restart a scan must already have a CID from when the scan was first started (and so presumably conglomCreated is true), but I have not verified this, so yes it might be a problem.

          Show
          dyret Dyre Tjeldvoll added a comment - Sigh... you are probably right. I guess that the safest thing to do is to stick with the CID==0 test, and set it to 0 whenever the temporary conglomerate is removed. But I think that a more explicit and intuitive predicate would have made the code easier to read. Come to think of it, I really loathe such "cooperating classes" where correct behavior depends on a shared state that's manipulated by both... but in this case I guess a smaller, less intrusive, change is preferable. Wrt. reStartScan(): I would assume that when you restart a scan must already have a CID from when the scan was first started (and so presumably conglomCreated is true), but I have not verified this, so yes it might be a problem.
          Hide
          knutanders Knut Anders Hatlen added a comment -

          I think the fix looks good, but I'm not completely convinced that (holder.CID == 0) is equivalent to (!holder.isConglomCreated()). In TemporaryRowHolderImpl.truncate(), conglomCreated is set to false, but CID is left untouched, so they may get out of sync. (But since truncate() first calls close(), which will drop the conglomerate, I don't think that this code will ever be executed. Perhaps it's better to remove it?) Similarly, TemporaryRowHolderResultSet.reStartScan() sets CID and leaves conglomCreated untouched. Do you think this can cause any problems?

          Show
          knutanders Knut Anders Hatlen added a comment - I think the fix looks good, but I'm not completely convinced that (holder.CID == 0) is equivalent to (!holder.isConglomCreated()). In TemporaryRowHolderImpl.truncate(), conglomCreated is set to false, but CID is left untouched, so they may get out of sync. (But since truncate() first calls close(), which will drop the conglomerate, I don't think that this code will ever be executed. Perhaps it's better to remove it?) Similarly, TemporaryRowHolderResultSet.reStartScan() sets CID and leaves conglomCreated untouched. Do you think this can cause any problems?
          Hide
          dyret Dyre Tjeldvoll added a comment -

          All tests pass (except jdbc4/ResultSetTest as reported in the Tinderbox test).

          Show
          dyret Dyre Tjeldvoll added a comment - All tests pass (except jdbc4/ResultSetTest as reported in the Tinderbox test).
          Hide
          dyret Dyre Tjeldvoll added a comment -

          I did some more digging, and I now think I know how it should be fixed, (creating a new TemporaryRowHolderImpl object in each execution works, but is overkill). I noticed that the problem did not occur EVERY time a prepared statement was re-executed. Sometimes it would not fail until the 9th execution.

          It turns out that the problem was a lack of communication between TemporaryRowHolderImpl and TemporaryRowHolderResultSet. When rows are inserted into the RowHolder it will fill as many rows as it can into its internal rowArray, before creating, and "spilling over" into a temporary conglomerate.

          The RowHolderResultSet is used to read the rows back out, so it starts by returning the rows from the rowArray, and then proceeds with rows from the temporary conglomerate. Unfortunately, the RowHolder doesn't provide a proper API for checking if a temporary conglomerate has actually been created. To get around this RowHolderResultSet checks if RowHolder.CID (the conglomerate id of the temporary conglomerate) is 0. If it is, RowHolderResultSet assumes that no temporary conglomerate has been created.

          This scheme worked well when internal result sets were not reused because then each execution started with a newly created RowHolder which had its CID member initialized to 0. But when results sets are reused this fact cannot be relied upon since CID it NOT set back to 0 by RowHolder.close().

          So the error will ONLY surface when a prepared statement has caused the rowHolder to spill into a temporary conglomerate (thereby assigning a value != 0 to CID), and some subsequent execution does NOT spill over. Then when the latter execution tries to read from the RowHolderResultSet it will incorrectly look for a temporary conglomerate that was never created in the insert-phase.

          I'm currently running tests on a patch that provides an api for testing if the conglomerate has been created (and also sets CID to 0 in close()). I plan to upload it if the tests pass.

          (There should probably be a regression test for this, but James' repro does not have ASF license so I guess we can't just convert that to JUnit).

          Show
          dyret Dyre Tjeldvoll added a comment - I did some more digging, and I now think I know how it should be fixed, (creating a new TemporaryRowHolderImpl object in each execution works, but is overkill). I noticed that the problem did not occur EVERY time a prepared statement was re-executed. Sometimes it would not fail until the 9th execution. It turns out that the problem was a lack of communication between TemporaryRowHolderImpl and TemporaryRowHolderResultSet. When rows are inserted into the RowHolder it will fill as many rows as it can into its internal rowArray, before creating, and "spilling over" into a temporary conglomerate. The RowHolderResultSet is used to read the rows back out, so it starts by returning the rows from the rowArray, and then proceeds with rows from the temporary conglomerate. Unfortunately, the RowHolder doesn't provide a proper API for checking if a temporary conglomerate has actually been created. To get around this RowHolderResultSet checks if RowHolder.CID (the conglomerate id of the temporary conglomerate) is 0. If it is, RowHolderResultSet assumes that no temporary conglomerate has been created. This scheme worked well when internal result sets were not reused because then each execution started with a newly created RowHolder which had its CID member initialized to 0. But when results sets are reused this fact cannot be relied upon since CID it NOT set back to 0 by RowHolder.close(). So the error will ONLY surface when a prepared statement has caused the rowHolder to spill into a temporary conglomerate (thereby assigning a value != 0 to CID), and some subsequent execution does NOT spill over. Then when the latter execution tries to read from the RowHolderResultSet it will incorrectly look for a temporary conglomerate that was never created in the insert-phase. I'm currently running tests on a patch that provides an api for testing if the conglomerate has been created (and also sets CID to 0 in close()). I plan to upload it if the tests pass. (There should probably be a regression test for this, but James' repro does not have ASF license so I guess we can't just convert that to JUnit).
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Minor update: Knut suggested that I could try to comment out the call to rowHolder.close() rather than the firstExecute test to make the code match the comment. Unfortunately, when I tried it, I got the missing conglomerate error again, so it seems that keeping the rowHolder open is not enough to prevent the temp conglomerate from disappearing.

          Show
          dyret Dyre Tjeldvoll added a comment - Minor update: Knut suggested that I could try to comment out the call to rowHolder.close() rather than the firstExecute test to make the code match the comment. Unfortunately, when I tried it, I got the missing conglomerate error again, so it seems that keeping the rowHolder open is not enough to prevent the temp conglomerate from disappearing.
          Hide
          dyret Dyre Tjeldvoll added a comment -

          The ASSERT reported in a debug build is really a red herring and
          not related to the original problem. I think the ASSERT simply is
          too strict as it does not allow what would be legal
          conversion. Simply modifying the queries so that they explicitly
          CAST all INTEGER values to BIGINT seems to remove that
          problem (see attached modified_SubShape.properties), and exposes the
          original problem with the missing temporary conglomerate. And
          that problem is only reported by the prepared statement that is
          actually re-executed, so that fits nicely with the DERBY-827
          hypothesis.

          The new call-stack from the line 57 (insert.test) statement shows
          that the missing conglomerate error comes from (re-)using
          InsertResultSet.rowHolder. At first I thought the problem was
          that close() isn't called on this object in
          InsertResultSet.close(), but then I noticed that
          normalInsertCore() calls close() on it:

          if (rowHolder != null)

          { rowHolder.close(); // rowHolder kept across opens }

          The comment doesn't really make sense since rowHolder, in fact,
          is closed. Anyway, the real problem is that code in the beginning
          of normalInsertCore() assumes that the comment is true, and only
          initializes rowHolder on the first execute:

          if (firstExecute && constants.deferred)

          { Properties properties = new Properties(); // Get the properties on the old heap rowChanger.getHeapConglomerateController().getInternalTablePropertySet(properties); /* ** If deferred we save a copy of the entire row. */ rowHolder = new TemporaryRowHolderImpl(activation, properties); rowChanger.setRowHolder(rowHolder); }

          Simply commenting out firstExecute in the if-test (see attached
          derby-3221.prelim.diff) makes the repro run without problems. I
          have not run any other tests with this change, nor do I know if
          this is the right approach. But at least it points to what causes
          the problem, and explains why it was introduced by DERBY-827.

          Show
          dyret Dyre Tjeldvoll added a comment - The ASSERT reported in a debug build is really a red herring and not related to the original problem. I think the ASSERT simply is too strict as it does not allow what would be legal conversion. Simply modifying the queries so that they explicitly CAST all INTEGER values to BIGINT seems to remove that problem (see attached modified_SubShape.properties), and exposes the original problem with the missing temporary conglomerate. And that problem is only reported by the prepared statement that is actually re-executed, so that fits nicely with the DERBY-827 hypothesis. The new call-stack from the line 57 (insert.test) statement shows that the missing conglomerate error comes from (re-)using InsertResultSet.rowHolder. At first I thought the problem was that close() isn't called on this object in InsertResultSet.close(), but then I noticed that normalInsertCore() calls close() on it: if (rowHolder != null) { rowHolder.close(); // rowHolder kept across opens } The comment doesn't really make sense since rowHolder, in fact, is closed. Anyway, the real problem is that code in the beginning of normalInsertCore() assumes that the comment is true, and only initializes rowHolder on the first execute: if (firstExecute && constants.deferred) { Properties properties = new Properties(); // Get the properties on the old heap rowChanger.getHeapConglomerateController().getInternalTablePropertySet(properties); /* ** If deferred we save a copy of the entire row. */ rowHolder = new TemporaryRowHolderImpl(activation, properties); rowChanger.setRowHolder(rowHolder); } Simply commenting out firstExecute in the if-test (see attached derby-3221.prelim.diff) makes the repro run without problems. I have not run any other tests with this change, nor do I know if this is the right approach. But at least it points to what causes the problem, and explains why it was introduced by DERBY-827 .
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Thanks for your feedback, Bryan and Knut. So it is indeed a consequence of DERBY-827

          Hmm, the strange part is that according to the call stack posted by Knut the error happens at SubShape.insert(SubShape.java:37):

          // if nothing is new, then nothing to do
          if (ps.executeUpdate() > 0) {

          // do pre stuff
          for (String stmt : sql.getProperty("pre"
          ).split(";"))

          { ps = conn.prepareStatement(stmt) ; ps.executeUpdate(); // <--- line 37 }

          So this particular statement is AFAICT never re-executed. So, in theory, DERBY-827 should not make a dfference here, and if I read James' comment correctly, it is the prepared statement being re-executed at line 57 that seems to be the culprit. Meaning that preparing it prior to each execute removes the symptom...

          So we have to assume then, that some cruft left in the internal result set tree of this prepared statement SOMEHOW causes the newly prepared statement at line 37 to fail. Right now, I cannot imagine how that could happen...

          Show
          dyret Dyre Tjeldvoll added a comment - Thanks for your feedback, Bryan and Knut. So it is indeed a consequence of DERBY-827 Hmm, the strange part is that according to the call stack posted by Knut the error happens at SubShape.insert(SubShape.java:37): // if nothing is new, then nothing to do if (ps.executeUpdate() > 0) { // do pre stuff for (String stmt : sql.getProperty("pre" ).split(";")) { ps = conn.prepareStatement(stmt) ; ps.executeUpdate(); // <--- line 37 } So this particular statement is AFAICT never re-executed. So, in theory, DERBY-827 should not make a dfference here, and if I read James' comment correctly, it is the prepared statement being re-executed at line 57 that seems to be the culprit. Meaning that preparing it prior to each execute removes the symptom... So we have to assume then, that some cruft left in the internal result set tree of this prepared statement SOMEHOW causes the newly prepared statement at line 37 to fail. Right now, I cannot imagine how that could happen...
          Hide
          knutanders Knut Anders Hatlen added a comment -

          Dyre, I think you are right that this is a regression caused by DERBY-827. I'm not able to reproduce the error on revision 540920, but it is reproducible on revision 540921, which introduced the reuse of result sets.

          Show
          knutanders Knut Anders Hatlen added a comment - Dyre, I think you are right that this is a regression caused by DERBY-827 . I'm not able to reproduce the error on revision 540920, but it is reproducible on revision 540921, which introduced the reuse of result sets.
          Hide
          bryanpendleton Bryan Pendleton added a comment -

          Hi Dyre, thanks for looking at this! You might find it useful to capture the generated
          code and decompile it, then look at that together with a close look at the query plan.
          Presumably the sort is due to a GROUP BY or ORDER BY clause in the query, or
          perhaps due to a DISTINCT? When I was looking at some datatype mismatch
          problems in ORDER BY processing (DERBY-2352), I saw that the code generation
          logic for these areas involves objects like the ResultColumn and the OrderByList.
          Hopefully this is helpful, let me know and I can try to offer some more ideas.

          Show
          bryanpendleton Bryan Pendleton added a comment - Hi Dyre, thanks for looking at this! You might find it useful to capture the generated code and decompile it, then look at that together with a close look at the query plan. Presumably the sort is due to a GROUP BY or ORDER BY clause in the query, or perhaps due to a DISTINCT? When I was looking at some datatype mismatch problems in ORDER BY processing ( DERBY-2352 ), I saw that the code generation logic for these areas involves objects like the ResultColumn and the OrderByList. Hopefully this is helpful, let me know and I can try to offer some more ideas.
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Looking more closely at the repro, I see that all columns are of type BIGINT. So it seems reasonable that the sort template has type SQLLongint, but why is it given a row where the 3rd (the failure is is reported for the third (colid=2) column) column has type SQLInteger?

          Show
          dyret Dyre Tjeldvoll added a comment - Looking more closely at the repro, I see that all columns are of type BIGINT. So it seems reasonable that the sort template has type SQLLongint, but why is it given a row where the 3rd (the failure is is reported for the third (colid=2) column) column has type SQLInteger?
          Hide
          dyret Dyre Tjeldvoll added a comment -

          My theory does NOT hold The failure still happens when sortTemplateRow is re-initialized in each call to openCore. Could this be a conversion thing? That is; should it be possible to promote an SQLInteger to an SQLLongint? If I read the code correctly, it seems like col2 is the sort template (SQLLongint) and col1 is from the current row. Also, where does the last NPE come from? Is that just a consequence, or is it significant?

          Show
          dyret Dyre Tjeldvoll added a comment - My theory does NOT hold The failure still happens when sortTemplateRow is re-initialized in each call to openCore. Could this be a conversion thing? That is; should it be possible to promote an SQLInteger to an SQLLongint? If I read the code correctly, it seems like col2 is the sort template (SQLLongint) and col1 is from the current row. Also, where does the last NPE come from? Is that just a consequence, or is it significant?
          Hide
          dyret Dyre Tjeldvoll added a comment -

          Could this be another followup error from DERBY-827? Seems like the problem COULD be that SortResultSet.sortTemplateRow is not reset/recomputed when SortResultSet is closed (nothing is done to it in SortResultSet.close()). It seems like the sanity check which compares sortTemplateRow to the row being fed to the sorter fails, which I guess could happen if the template left over from the previous execution doesn't match the new one.

          I'll try out the repro and see if the theory holds.

          Show
          dyret Dyre Tjeldvoll added a comment - Could this be another followup error from DERBY-827 ? Seems like the problem COULD be that SortResultSet.sortTemplateRow is not reset/recomputed when SortResultSet is closed (nothing is done to it in SortResultSet.close()). It seems like the sanity check which compares sortTemplateRow to the row being fed to the sorter fails, which I guess could happen if the template left over from the previous execution doesn't match the new one. I'll try out the repro and see if the theory holds.
          Hide
          knutanders Knut Anders Hatlen added a comment -

          When I run the test case with a sane build (trunk), I get an assert failure:

          ij> INSERT INTO SubShape (nID, nSubID) SELECT nID, nSubID FROM Test;
          java.sql.SQLException: Java exception: 'ASSERT FAILED col1.getClass() (class org.apache.derby.iapi.types.SQLInteger) expected to be the same as col2.getClass() (class org.apache.derby.iapi.types.SQLLongint): org.apache.derby.shared.common.sanity.AssertFailure'.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:95)
          at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:88)
          at org.apache.derby.impl.jdbc.Util.javaException(Util.java:245)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:403)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:346)
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:1666)
          at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:81)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1324)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(EmbedPreparedStatement.java:1650)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(EmbedPreparedStatement.java:299)
          at SubShape.insert(SubShape.java:37)
          at org.apache.derby.exe.ac12564092x0117x3b06x55cfx0000003a57281.g0(Unknown Source)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(ReflectMethod.java:46)
          at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(CallStatementResultSet.java:74)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:370)
          at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(GenericTriggerExecutor.java:159)
          at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(StatementTriggerExecutor.java:80)
          at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(TriggerEventActivator.java:269)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(InsertResultSet.java:1150)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:487)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:370)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1234)
          at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:624)
          at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:556)
          at org.apache.derby.impl.tools.ij.ij.executeImmediate(ij.java:330)
          at org.apache.derby.impl.tools.ij.utilMain.doCatch(utilMain.java:508)
          at org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(utilMain.java:350)
          at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:248)
          at org.apache.derby.impl.tools.ij.Main.go(Main.java:215)
          at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181)
          at org.apache.derby.impl.tools.ij.Main.main(Main.java:73)
          at org.apache.derby.tools.ij.main(ij.java:59)
          Caused by: java.sql.SQLException: Java exception: 'ASSERT FAILED col1.getClass() (class org.apache.derby.iapi.types.SQLInteger) expected to be the same as col2.getClass() (class org.apache.derby.iapi.types.SQLLongint): org.apache.derby.shared.common.sanity.AssertFailure'.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:135)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:70)
          ... 35 more
          Caused by: org.apache.derby.shared.common.sanity.AssertFailure: ASSERT FAILED col1.getClass() (class org.apache.derby.iapi.types.SQLInteger) expected to be the same as col2.getClass() (class org.apache.derby.iapi.types.SQLLongint)
          at org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:162)
          at org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:147)
          at org.apache.derby.impl.store.access.sort.MergeSort.checkColumnTypes(MergeSort.java:472)
          at org.apache.derby.impl.store.access.sort.MergeInserter.insert(MergeInserter.java:98)
          at org.apache.derby.impl.sql.execute.SortResultSet.loadSorter(SortResultSet.java:317)
          at org.apache.derby.impl.sql.execute.SortResultSet.openCore(SortResultSet.java:268)
          at org.apache.derby.impl.sql.execute.NormalizeResultSet.openCore(NormalizeResultSet.java:147)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:407)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:370)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1234)
          ... 28 more
          ERROR XJ001: Java exception: ': java.lang.NullPointerException'.

          Show
          knutanders Knut Anders Hatlen added a comment - When I run the test case with a sane build (trunk), I get an assert failure: ij> INSERT INTO SubShape (nID, nSubID) SELECT nID, nSubID FROM Test; java.sql.SQLException: Java exception: 'ASSERT FAILED col1.getClass() (class org.apache.derby.iapi.types.SQLInteger) expected to be the same as col2.getClass() (class org.apache.derby.iapi.types.SQLLongint): org.apache.derby.shared.common.sanity.AssertFailure'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:95) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:88) at org.apache.derby.impl.jdbc.Util.javaException(Util.java:245) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:403) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:346) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:1666) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:81) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1324) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(EmbedPreparedStatement.java:1650) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(EmbedPreparedStatement.java:299) at SubShape.insert(SubShape.java:37) at org.apache.derby.exe.ac12564092x0117x3b06x55cfx0000003a57281.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(ReflectMethod.java:46) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(CallStatementResultSet.java:74) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:370) at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(GenericTriggerExecutor.java:159) at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(StatementTriggerExecutor.java:80) at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(TriggerEventActivator.java:269) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(InsertResultSet.java:1150) at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:487) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:370) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1234) at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:624) at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:556) at org.apache.derby.impl.tools.ij.ij.executeImmediate(ij.java:330) at org.apache.derby.impl.tools.ij.utilMain.doCatch(utilMain.java:508) at org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(utilMain.java:350) at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:248) at org.apache.derby.impl.tools.ij.Main.go(Main.java:215) at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:181) at org.apache.derby.impl.tools.ij.Main.main(Main.java:73) at org.apache.derby.tools.ij.main(ij.java:59) Caused by: java.sql.SQLException: Java exception: 'ASSERT FAILED col1.getClass() (class org.apache.derby.iapi.types.SQLInteger) expected to be the same as col2.getClass() (class org.apache.derby.iapi.types.SQLLongint): org.apache.derby.shared.common.sanity.AssertFailure'. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:135) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:70) ... 35 more Caused by: org.apache.derby.shared.common.sanity.AssertFailure: ASSERT FAILED col1.getClass() (class org.apache.derby.iapi.types.SQLInteger) expected to be the same as col2.getClass() (class org.apache.derby.iapi.types.SQLLongint) at org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:162) at org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:147) at org.apache.derby.impl.store.access.sort.MergeSort.checkColumnTypes(MergeSort.java:472) at org.apache.derby.impl.store.access.sort.MergeInserter.insert(MergeInserter.java:98) at org.apache.derby.impl.sql.execute.SortResultSet.loadSorter(SortResultSet.java:317) at org.apache.derby.impl.sql.execute.SortResultSet.openCore(SortResultSet.java:268) at org.apache.derby.impl.sql.execute.NormalizeResultSet.openCore(NormalizeResultSet.java:147) at org.apache.derby.impl.sql.execute.InsertResultSet.open(InsertResultSet.java:407) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:370) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1234) ... 28 more ERROR XJ001: Java exception: ': java.lang.NullPointerException'.
          Hide
          northshorefiend James Alan Shepherd added a comment -

          If I don't reuse the prepared statement, then the bug is avoided.

          SubShape.java:

          for (long i = r.getLong(1); i >= 0; i--) {
          if (conn.prepareStatement(sql
          .getProperty("insert.test")).executeUpdate() < 1)
          break;

          So, I guess this bug is caused by something not being cleared when reusing the PreparedStatement.

          A work around is basically to not use the PreparedStatement again, which obviously defeats the purpose

          Show
          northshorefiend James Alan Shepherd added a comment - If I don't reuse the prepared statement, then the bug is avoided. SubShape.java: for (long i = r.getLong(1); i >= 0; i--) { if (conn.prepareStatement(sql .getProperty("insert.test")).executeUpdate() < 1) break; So, I guess this bug is caused by something not being cleared when reusing the PreparedStatement. A work around is basically to not use the PreparedStatement again, which obviously defeats the purpose
          Hide
          northshorefiend James Alan Shepherd added a comment -

          Tried with different test data and still hit the bug, so here's a test case.

          Includes a stored proc, create script, data, etc.

          With derby on the CLASSPATH should reproduce just fine with 10.3.2.1 and not fail with 10.2.2.0

          I have reduced things as far as I could. Don't even have constraints now. Only 3 tables with 2 cols each.

          It seems the 'insert.first' query in SubShape.properties is necessary to reproduce the bug, so may be it has something to do with updating a table you have already updated in the same tran?

          And it seems to be when the sp is called for the second time? but I'm pretty sure there is no recursion problem there. Anyhoo, it works fine on 10.2.2.0, so I don't think the trigger/sp is doing anything too strange.

          Show
          northshorefiend James Alan Shepherd added a comment - Tried with different test data and still hit the bug, so here's a test case. Includes a stored proc, create script, data, etc. With derby on the CLASSPATH should reproduce just fine with 10.3.2.1 and not fail with 10.2.2.0 I have reduced things as far as I could. Don't even have constraints now. Only 3 tables with 2 cols each. It seems the 'insert.first' query in SubShape.properties is necessary to reproduce the bug, so may be it has something to do with updating a table you have already updated in the same tran? And it seems to be when the sp is called for the second time? but I'm pretty sure there is no recursion problem there. Anyhoo, it works fine on 10.2.2.0, so I don't think the trigger/sp is doing anything too strange.
          Hide
          northshorefiend James Alan Shepherd added a comment -

          I'm going in....

          rolling my own 10.3.2.1...

          Opening impl/sql/execute/TemporaryRowHolderImpl.java....

          public static final int DEFAULT_OVERFLOWTHRESHOLD = 100;

          squashes the bug - well, probably just chases it into the undergrowth.

          I'll probably fiddle the test data that's used on Monday to avoid the bug

          Show
          northshorefiend James Alan Shepherd added a comment - I'm going in.... rolling my own 10.3.2.1... Opening impl/sql/execute/TemporaryRowHolderImpl.java.... public static final int DEFAULT_OVERFLOWTHRESHOLD = 100; squashes the bug - well, probably just chases it into the undergrowth. I'll probably fiddle the test data that's used on Monday to avoid the bug
          Hide
          northshorefiend James Alan Shepherd added a comment -

          ooh, 10.3.2.1 is out:

          Bug still present

          Show
          northshorefiend James Alan Shepherd added a comment - ooh, 10.3.2.1 is out: Bug still present
          Hide
          army A B added a comment -

          Ah, never mind, I see Mike already covered what I just wrote:

          "...looks like we are somehow losing the temporary file associated with a deferred update"

          Sorry for the noise (oops).

          Show
          army A B added a comment - Ah, never mind, I see Mike already covered what I just wrote: "...looks like we are somehow losing the temporary file associated with a deferred update" Sorry for the noise (oops).
          Hide
          army A B added a comment - - edited

          > I can narrow it down to the inclusion of five rows in the selected set - these rows don't seem to
          > cause any different logic to be followed

          I haven't been following this very closely, but the fact that 5 rows is the magic number and that Mike mentioned temporary files made me think of impl/sql/execute/TemporaryRowHolderImpl.java, where we have:

          public static final int DEFAULT_OVERFLOWTHRESHOLD = 5;

          Examination of the code shows that this will be the size of "rowArray" for the TemporaryRowHolder, and once we reach 5 rows we'll create a temporary conglomerate to store the extra rows. I can't be certain, but I think that one of the ways in which we use TemporaryRowHolders is when we update a unique index column, in which case we put the new row into a temporary row holder, delete the old row, then insert from the temporary row holder. This indirection allows correct processing of scenarios when the new (updated) value of one row matches the old value of another row (by deleting the old row first and then inserting new row, we avoid throwing a duplicate key violation).

          Maybe that's related, maybe not. Just thought I'd mention it since I saw the number 5...

          Show
          army A B added a comment - - edited > I can narrow it down to the inclusion of five rows in the selected set - these rows don't seem to > cause any different logic to be followed I haven't been following this very closely, but the fact that 5 rows is the magic number and that Mike mentioned temporary files made me think of impl/sql/execute/TemporaryRowHolderImpl.java, where we have: public static final int DEFAULT_OVERFLOWTHRESHOLD = 5; Examination of the code shows that this will be the size of "rowArray" for the TemporaryRowHolder, and once we reach 5 rows we'll create a temporary conglomerate to store the extra rows. I can't be certain, but I think that one of the ways in which we use TemporaryRowHolders is when we update a unique index column, in which case we put the new row into a temporary row holder, delete the old row, then insert from the temporary row holder. This indirection allows correct processing of scenarios when the new (updated) value of one row matches the old value of another row (by deleting the old row first and then inserting new row, we avoid throwing a duplicate key violation). Maybe that's related, maybe not. Just thought I'd mention it since I saw the number 5...
          Hide
          northshorefiend James Alan Shepherd added a comment -

          I can narrow it down to the inclusion of five rows in the selected set - these rows don't seem to cause any different logic to be followed, but they possibly cause a loop that executes SQL in the proc to be run a few extra times.

          For now, I'm going back to 10.2.2.0. Everything works there.

          May be in the future a stupid mistake in the script will show itself.

          Last thought: would a shorter transaction make such a conglomerate error less likely? Though from a business logic point of view this isn't an option.

          Show
          northshorefiend James Alan Shepherd added a comment - I can narrow it down to the inclusion of five rows in the selected set - these rows don't seem to cause any different logic to be followed, but they possibly cause a loop that executes SQL in the proc to be run a few extra times. For now, I'm going back to 10.2.2.0. Everything works there. May be in the future a stupid mistake in the script will show itself. Last thought: would a shorter transaction make such a conglomerate error less likely? Though from a business logic point of view this isn't an option.
          Hide
          northshorefiend James Alan Shepherd added a comment -

          Yes it's pretty deep in a trigger.

          I think the statement that kicks everything off, that you request is

          INSERT INTO SubTTT ( nShapeID , nSubShapeID , nColorID ) SELECT DISTINCT C.nRCID , S.nRCID , (S
          ELECT nColorID FROM Colorition WHERE nIsCCC = 1) FROM LegacySubCCC SC JOIN LegacyCCC C ON SC.nRunID = ? AND C.nRunID = ? AND C.nICCCID = SC.nICCCID JOIN LegacyCCC S ON S.nRunID = ? AND S.nICCCID = SC.nISubCCCID LEF
          T JOIN SubTTT SO ON SO.nShapeID = C.nRCID AND SO.nSubShapeID = S.nRCID WHERE SO.nShapeID IS NULL

          which is logged up there somewhere.

          There are no errors before what I have posted, but the execution of the above statement is several pages back in the log.

          As you say, changing the code a bit stops the bug. Interestingly, if I select out of a different Legacy* table then I don't get the bug.

          Even more interestingly, if I shove the data in a Temporary table, then select out of that, I get the bug still! And the conglomerate number changes.

          Absolutely 100% the same each time I run the whole shebang.

          Not really multi-threaded, there is an ij session on stdout/in so that I can take a peek in the embedded database while debugging. However, I don't think this connects before the bug bites. The java app does try to connect to an existing database, and if this fails, connect again with ;create=true, and runs the create script. Actually, initially, there may be another connection opened adn closed that just finds out what version of derby is running etc, but this closes straight after having a look around.

          I'm just trying to run with fewer rows in the table that gets selected from, for the hell of it, as the bug arises with the largest row count Legacy table.
          Initial results seem to show that fewer rows (we're only talking 4 being few, 100 being many) doesn't trigger the bug. I'm gonna check that this still exercises the same code path.

          Show
          northshorefiend James Alan Shepherd added a comment - Yes it's pretty deep in a trigger. I think the statement that kicks everything off, that you request is INSERT INTO SubTTT ( nShapeID , nSubShapeID , nColorID ) SELECT DISTINCT C.nRCID , S.nRCID , (S ELECT nColorID FROM Colorition WHERE nIsCCC = 1) FROM LegacySubCCC SC JOIN LegacyCCC C ON SC.nRunID = ? AND C.nRunID = ? AND C.nICCCID = SC.nICCCID JOIN LegacyCCC S ON S.nRunID = ? AND S.nICCCID = SC.nISubCCCID LEF T JOIN SubTTT SO ON SO.nShapeID = C.nRCID AND SO.nSubShapeID = S.nRCID WHERE SO.nShapeID IS NULL which is logged up there somewhere. There are no errors before what I have posted, but the execution of the above statement is several pages back in the log. As you say, changing the code a bit stops the bug. Interestingly, if I select out of a different Legacy* table then I don't get the bug. Even more interestingly, if I shove the data in a Temporary table, then select out of that, I get the bug still! And the conglomerate number changes. Absolutely 100% the same each time I run the whole shebang. Not really multi-threaded, there is an ij session on stdout/in so that I can take a peek in the embedded database while debugging. However, I don't think this connects before the bug bites. The java app does try to connect to an existing database, and if this fails, connect again with ;create=true, and runs the create script. Actually, initially, there may be another connection opened adn closed that just finds out what version of derby is running etc, but this closes straight after having a look around. I'm just trying to run with fewer rows in the table that gets selected from, for the hell of it, as the bug arises with the largest row count Legacy table. Initial results seem to show that fewer rows (we're only talking 4 being few, 100 being many) doesn't trigger the bug. I'm gonna check that this still exercises the same code path.
          Hide
          mikem Mike Matrigali added a comment -

          looking at your recent posting I think the error is actually while executing code as part of a trigger.
          Unfortunately I don't think are error logging handles this nesting of execution very well. The error is happening
          as part of executing code as part of a trigger - but I don't think we are logging the query in the trigger that is
          causing the problem.

          Would it be possible to post the query that is executing in your java code. :
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43)
          at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80)
          at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107)

          Is it possible that code has changed somewhat and that could be what affects whether the bug reproduces or not?

          Show
          mikem Mike Matrigali added a comment - looking at your recent posting I think the error is actually while executing code as part of a trigger. Unfortunately I don't think are error logging handles this nesting of execution very well. The error is happening as part of executing code as part of a trigger - but I don't think we are logging the query in the trigger that is causing the problem. Would it be possible to post the query that is executing in your java code. : at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43) at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80) at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107) Is it possible that code has changed somewhat and that could be what affects whether the bug reproduces or not?
          Hide
          mikem Mike Matrigali added a comment -

          in the case where you can reproduce this. Is it always the same statement at the same point in the app that fails? Is the error always exactly the same (ie. is the conglomerate number always -15)?

          Show
          mikem Mike Matrigali added a comment - in the case where you can reproduce this. Is it always the same statement at the same point in the app that fails? Is the error always exactly the same (ie. is the conglomerate number always -15)?
          Hide
          mikem Mike Matrigali added a comment -

          What is the last error, if any in derby.log which precedes the error you posted.

          Also I see that this is session 2. Is your app multi-threaded? Is it using multiple connections?

          Show
          mikem Mike Matrigali added a comment - What is the last error, if any in derby.log which precedes the error you posted. Also I see that this is session 2. Is your app multi-threaded? Is it using multiple connections?
          Hide
          northshorefiend James Alan Shepherd added a comment -

          Well, of course, I switch back to 10.3.1.4 today, and no bug!!! So as I thought, test case; not so much.

          BUT, with the wonder that is the Eclipse local history, and the mystery that is svn, once again the error arises.

          I can't publish the full log (956k) as it contains a year's work that hasn't been paid for yet I realise that this might not go back far enough, it's the best I can do, apologies. Still, at least anyone that comes across such things will know that they aren't delusional.

          I did try 'sync && sync' before firing up the java code, but it made no difference. The filesystem all this is happening on has recently changed to being a linux software raid, but other than that everything else is pretty vanilla ext3 with nothing else accessing the dir.

          derby.properties now reads:
          -----8<----------
          derby.stream.error.logSeverityLevel=0
          derby.language.logStatementText=true
          derby.infolog.append=true
          -----8<----------

          So, we are in a stored proc fired by a trigger. (names may have been changed to pretext the innocent)

          2007-12-12 13:22:40.326 GMT Thread[Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Executing prepared statement: INSERT INTO SubTTTXX ( nShapeID , nSubShapeID , nColorID , nHelp ) SELECT DISTINCT X.nColorID
          , DSO.nSubShapeID , X.nColorID , (SELECT INTEGER(MAX(nHelp)+1) FROM SubTTTXX) FROM SubTTTXX X JOIN DirectSubTTT DSO ON X.nHelp > 0 AND DSO.nShapeID = X.nSubShapeID LEFT JOIN SubTTT SO ON SO.nShapeID = X.nShapeID AND SO.nSubShapeID = DSO.nSu
          bShapeID WHERE X.nShapeID <> DSO.nSubShapeID AND SO.nShapeID IS NULL :End prepared statement
          2007-12-12 13:22:40.330 GMT Thread[Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Cleanup action starting
          2007-12-12 13:22:40.330 GMT Thread[Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Failed Statement is: INSERT INTO SubTTTXX ( nShapeID , nSubShapeID , nColorID , nHelp ) SELECT DISTINCT X.nShapeID , DSO
          .nSubShapeID , X.nColorID , (SELECT INTEGER(MAX(nHelp)+1) FROM SubTTTXX) FROM SubTTTXX X JOIN DirectSubTTT DSO ON X.nHelp > 0 AND DSO.nShapeID = X.nSubShapeID LEFT JOIN SubTTT SO ON SO.nShapeID = X.nShapeID AND SO.nSubShapeID = DSO.nSubShapeID WHE
          RE X.nShapeID <> DSO.nSubShapeID AND SO.nShapeID IS NULL
          ERROR XSAI2: The conglomerate (-15) requested does not exist.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43)
          at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80)
          at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107)
          at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
          at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source)
          at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source)
          at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745)
          at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801)
          at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218)
          at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383)
          at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94)
          at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352)
          at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
          at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134)
          at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339)
          at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44)
          at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57)
          at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208)
          at com.xxx.zzz.kkk.Start.startup(Start.java:160)
          at com.xxx.start.Starter.startup(Starter.java:264)
          at com.xxx.start.Main.startup(Main.java:270)
          at com.xxx.start.Main.main(Main.java:199)
          Cleanup action completed
          2007-12-12 13:22:40.490 GMT Thread[Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Cleanup action starting
          2007-12-12 13:22:40.490 GMT Thread[Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Failed Statement is: INSERT INTO SubTTT ( nShapeID , nSubShapeID , nColorID ) SELECT DISTINCT C.nRCID , S.nRCID , (S
          ELECT nColorID FROM Colorition WHERE nIsCCC = 1) FROM LegacySubCCC SC JOIN LegacyCCC C ON SC.nRunID = ? AND C.nRunID = ? AND C.nICCCID = SC.nICCCID JOIN LegacyCCC S ON S.nRunID = ? AND S.nICCCID = SC.nISubCCCID LEF
          T JOIN SubTTT SO ON SO.nShapeID = C.nRCID AND SO.nSubShapeID = S.nRCID WHERE SO.nShapeID IS NULL with 3 parameters begin parameter #1: 1 :end parameter begin parameter #2: 1 :end parameter begin parameter #3: 1 :end parameter
          ERROR 38000: The exception 'java.sql.SQLException: The conglomerate (-15) requested does not exist.' was thrown while evaluating an expression.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown Source)
          at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
          at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source)
          at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source)
          at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745)
          at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801)
          at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218)
          at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383)
          at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94)
          at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352)
          at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
          at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134)
          at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339)
          at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44)
          at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57)
          at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208)
          at com.xxx.zzz.kkk.Start.startup(Start.java:160)
          at com.xxx.start.Starter.startup(Starter.java:264)
          at com.xxx.start.Main.startup(Main.java:270)
          at com.xxx.start.Main.main(Main.java:199)
          Caused by: java.sql.SQLException: The conglomerate (-15) requested does not exist.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43)
          at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80)
          at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107)
          at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          ... 32 more
          Caused by: java.sql.SQLException: The conglomerate (-15) requested does not exist.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
          ... 50 more
          Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          ... 44 more
          ============= begin nested exception, level (1) ===========
          java.sql.SQLException: The conglomerate (-15) requested does not exist.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43)
          at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80)
          at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107)
          at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
          at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source)
          at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source)
          at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745)
          at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801)
          at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218)
          at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383)
          at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94)
          at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352)
          at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
          at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134)
          at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339)
          at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44)
          at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57)
          at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208)
          at com.xxx.zzz.kkk.Start.startup(Start.java:160)
          at com.xxx.start.Starter.startup(Starter.java:264)
          at com.xxx.start.Main.startup(Main.java:270)
          at com.xxx.start.Main.main(Main.java:199)
          Caused by: java.sql.SQLException: The conglomerate (-15) requested does not exist.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
          ... 50 more
          Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          ... 44 more
          ============= end nested exception, level (1) ===========
          ============= begin nested exception, level (2) ===========
          java.sql.SQLException: The conglomerate (-15) requested does not exist.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43)
          at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80)
          at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107)
          at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
          at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source)
          at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source)
          at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745)
          at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801)
          at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218)
          at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383)
          at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94)
          at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352)
          at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
          at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134)
          at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339)
          at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44)
          at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57)
          at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208)
          at com.xxx.zzz.kkk.Start.startup(Start.java:160)
          at com.xxx.start.Starter.startup(Starter.java:264)
          at com.xxx.start.Main.startup(Main.java:270)
          at com.xxx.start.Main.main(Main.java:199)
          Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          ... 44 more
          ============= end nested exception, level (2) ===========
          ============= begin nested exception, level (3) ===========
          ERROR XSAI2: The conglomerate (-15) requested does not exist.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79)
          at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43)
          at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80)
          at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107)
          at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
          at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source)
          at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source)
          at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
          at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
          at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745)
          at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797)
          at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801)
          at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218)
          at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383)
          at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94)
          at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352)
          at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
          at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134)
          at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339)
          at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44)
          at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57)
          at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208)
          at com.xxx.zzz.kkk.Start.startup(Start.java:160)
          at com.xxx.start.Starter.startup(Starter.java:264)
          at com.xxx.start.Main.startup(Main.java:270)
          at com.xxx.start.Main.main(Main.java:199)
          ============= end nested exception, level (3) ===========
          Cleanup action completed
          2007-12-12 13:22:40.524 GMT Thread[Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DATABASE = directory:shape), (DRDAID = null), Rolling back
          2007-12-12 13:22:40.885 GMT Thread[Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Committing
          2007-12-12 13:22:40.893 GMT Thread[xxx-start ShutdownHook,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DATABASE = directory:shape), (DRDAID = null), Rolling back

          2007-12-12 13:22:43.104 GMT:
          Shutting down instance c013800d-0116-ce84-f62d-000055b50d04


          (note2self: I have a patch against r192 that reproduces this, and a full copy of the derby.log)

          Show
          northshorefiend James Alan Shepherd added a comment - Well, of course, I switch back to 10.3.1.4 today, and no bug!!! So as I thought, test case; not so much. BUT, with the wonder that is the Eclipse local history, and the mystery that is svn, once again the error arises. I can't publish the full log (956k) as it contains a year's work that hasn't been paid for yet I realise that this might not go back far enough, it's the best I can do, apologies. Still, at least anyone that comes across such things will know that they aren't delusional. I did try 'sync && sync' before firing up the java code, but it made no difference. The filesystem all this is happening on has recently changed to being a linux software raid, but other than that everything else is pretty vanilla ext3 with nothing else accessing the dir. derby.properties now reads: ----- 8< ---------- derby.stream.error.logSeverityLevel=0 derby.language.logStatementText=true derby.infolog.append=true ----- 8< ---------- So, we are in a stored proc fired by a trigger. (names may have been changed to pretext the innocent) 2007-12-12 13:22:40.326 GMT Thread [Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Executing prepared statement: INSERT INTO SubTTTXX ( nShapeID , nSubShapeID , nColorID , nHelp ) SELECT DISTINCT X.nColorID , DSO.nSubShapeID , X.nColorID , (SELECT INTEGER(MAX(nHelp)+1) FROM SubTTTXX) FROM SubTTTXX X JOIN DirectSubTTT DSO ON X.nHelp > 0 AND DSO.nShapeID = X.nSubShapeID LEFT JOIN SubTTT SO ON SO.nShapeID = X.nShapeID AND SO.nSubShapeID = DSO.nSu bShapeID WHERE X.nShapeID <> DSO.nSubShapeID AND SO.nShapeID IS NULL :End prepared statement 2007-12-12 13:22:40.330 GMT Thread [Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Cleanup action starting 2007-12-12 13:22:40.330 GMT Thread [Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Failed Statement is: INSERT INTO SubTTTXX ( nShapeID , nSubShapeID , nColorID , nHelp ) SELECT DISTINCT X.nShapeID , DSO .nSubShapeID , X.nColorID , (SELECT INTEGER(MAX(nHelp)+1) FROM SubTTTXX) FROM SubTTTXX X JOIN DirectSubTTT DSO ON X.nHelp > 0 AND DSO.nShapeID = X.nSubShapeID LEFT JOIN SubTTT SO ON SO.nShapeID = X.nShapeID AND SO.nSubShapeID = DSO.nSubShapeID WHE RE X.nShapeID <> DSO.nSubShapeID AND SO.nShapeID IS NULL ERROR XSAI2: The conglomerate (-15) requested does not exist. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43) at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80) at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107) at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source) at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801) at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218) at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383) at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94) at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352) at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134) at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339) at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44) at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57) at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208) at com.xxx.zzz.kkk.Start.startup(Start.java:160) at com.xxx.start.Starter.startup(Starter.java:264) at com.xxx.start.Main.startup(Main.java:270) at com.xxx.start.Main.main(Main.java:199) Cleanup action completed 2007-12-12 13:22:40.490 GMT Thread [Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Cleanup action starting 2007-12-12 13:22:40.490 GMT Thread [Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Failed Statement is: INSERT INTO SubTTT ( nShapeID , nSubShapeID , nColorID ) SELECT DISTINCT C.nRCID , S.nRCID , (S ELECT nColorID FROM Colorition WHERE nIsCCC = 1) FROM LegacySubCCC SC JOIN LegacyCCC C ON SC.nRunID = ? AND C.nRunID = ? AND C.nICCCID = SC.nICCCID JOIN LegacyCCC S ON S.nRunID = ? AND S.nICCCID = SC.nISubCCCID LEF T JOIN SubTTT SO ON SO.nShapeID = C.nRCID AND SO.nSubShapeID = S.nRCID WHERE SO.nShapeID IS NULL with 3 parameters begin parameter #1: 1 :end parameter begin parameter #2: 1 :end parameter begin parameter #3: 1 :end parameter ERROR 38000: The exception 'java.sql.SQLException: The conglomerate (-15) requested does not exist.' was thrown while evaluating an expression. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown Source) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source) at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801) at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218) at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383) at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94) at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352) at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134) at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339) at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44) at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57) at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208) at com.xxx.zzz.kkk.Start.startup(Start.java:160) at com.xxx.start.Starter.startup(Starter.java:264) at com.xxx.start.Main.startup(Main.java:270) at com.xxx.start.Main.main(Main.java:199) Caused by: java.sql.SQLException: The conglomerate (-15) requested does not exist. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43) at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80) at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107) at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) ... 32 more Caused by: java.sql.SQLException: The conglomerate (-15) requested does not exist. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) ... 50 more Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) ... 44 more ============= begin nested exception, level (1) =========== java.sql.SQLException: The conglomerate (-15) requested does not exist. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43) at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80) at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107) at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source) at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801) at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218) at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383) at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94) at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352) at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134) at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339) at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44) at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57) at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208) at com.xxx.zzz.kkk.Start.startup(Start.java:160) at com.xxx.start.Starter.startup(Starter.java:264) at com.xxx.start.Main.startup(Main.java:270) at com.xxx.start.Main.main(Main.java:199) Caused by: java.sql.SQLException: The conglomerate (-15) requested does not exist. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) ... 50 more Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) ... 44 more ============= end nested exception, level (1) =========== ============= begin nested exception, level (2) =========== java.sql.SQLException: The conglomerate (-15) requested does not exist. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43) at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80) at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107) at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source) at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801) at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218) at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383) at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94) at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352) at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134) at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339) at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44) at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57) at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208) at com.xxx.zzz.kkk.Start.startup(Start.java:160) at com.xxx.start.Starter.startup(Starter.java:264) at com.xxx.start.Main.startup(Main.java:270) at com.xxx.start.Main.main(Main.java:199) Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) ... 44 more ============= end nested exception, level (2) =========== ============= begin nested exception, level (3) =========== ERROR XSAI2: The conglomerate (-15) requested does not exist. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:79) at com.xxx.zzz.kkk.sp.SubTTT$1.work(SubTTT.java:43) at com.xxx.zzz.kkk.sp.Template.getResult(Template.java:80) at com.xxx.zzz.kkk.sp.SubTTT.insert(SubTTT.java:107) at org.apache.derby.exe.ac12564092x0116xce84xf62dx000055b50d0421.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown Source) at org.apache.derby.impl.sql.execute.StatementTriggerExecutor.fireTrigger(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101) at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:745) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:538) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:739) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:797) at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:801) at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:218) at com.xxx.yyy.DbOOOFactory$DBOOO.importFromLegacyTables(DbOOOFactory.java:383) at com.xxx.yyy.DbOOOFactory$DBOOO.access$1500(DbOOOFactory.java:94) at com.xxx.yyy.DbOOOFactory$DBOOO$2.doInTransactionWithoutResult(DbOOOFactory.java:352) at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:134) at com.xxx.yyy.DbOOOFactory$DBOOO.add(DbOOOFactory.java:339) at com.xxx.zzz.kkk.MetaJJJ.reload(MetaJJJ.java:44) at com.xxx.zzz.kkk.MetaJJJ.startup(MetaJJJ.java:57) at com.xxx.zzz.kkk.Start.startupZzz(Start.java:208) at com.xxx.zzz.kkk.Start.startup(Start.java:160) at com.xxx.start.Starter.startup(Starter.java:264) at com.xxx.start.Main.startup(Main.java:270) at com.xxx.start.Main.main(Main.java:199) ============= end nested exception, level (3) =========== Cleanup action completed 2007-12-12 13:22:40.524 GMT Thread [Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DATABASE = directory:shape), (DRDAID = null), Rolling back 2007-12-12 13:22:40.885 GMT Thread [Main,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DRDAID = null), Committing 2007-12-12 13:22:40.893 GMT Thread [xxx-start ShutdownHook,5,main] (XID = 1452), (SESSIONID = 2), (DATABASE = directory:shape), (DATABASE = directory:shape), (DRDAID = null), Rolling back 2007-12-12 13:22:43.104 GMT: Shutting down instance c013800d-0116-ce84-f62d-000055b50d04 (note2self: I have a patch against r192 that reproduces this, and a full copy of the derby.log)
          Hide
          mikem Mike Matrigali added a comment -

          Negative conglomerate numbers are used for temporary files. Are you doing anything special with temporary
          files in your environment - ie. possibly setting derby option to relocate them? Any chance some other process
          is going after temporary files while your app is still working? From minimal info available it looks like we are
          somehow losing the temporary file associated with a deferred update, but I could be reading the incomplete
          stack wrong.

          Best chance for someone to fix this is if you can post a reproducible case that a developer could run in their
          own environment.

          Since you have a 100% reproducible case, if you can't do that can you do the following to give a clue what is
          going on when the error is encountered:

          o start with a new database
          o set option so that we never lose anything from the error log file: derby.infolog.append=true
          http://db.apache.org/derby/docs/10.3/tuning/rtunproper13217.html
          o set option to log the sql that you are doing to derby.log: derby.language.logStatementText=true
          http://db.apache.org/derby/docs/10.3/tuning/rtunproper43517.html
          reproduce bug and post COMPLETE derby.log from test run.

          Show
          mikem Mike Matrigali added a comment - Negative conglomerate numbers are used for temporary files. Are you doing anything special with temporary files in your environment - ie. possibly setting derby option to relocate them? Any chance some other process is going after temporary files while your app is still working? From minimal info available it looks like we are somehow losing the temporary file associated with a deferred update, but I could be reading the incomplete stack wrong. Best chance for someone to fix this is if you can post a reproducible case that a developer could run in their own environment. Since you have a 100% reproducible case, if you can't do that can you do the following to give a clue what is going on when the error is encountered: o start with a new database o set option so that we never lose anything from the error log file: derby.infolog.append=true http://db.apache.org/derby/docs/10.3/tuning/rtunproper13217.html o set option to log the sql that you are doing to derby.log: derby.language.logStatementText=true http://db.apache.org/derby/docs/10.3/tuning/rtunproper43517.html reproduce bug and post COMPLETE derby.log from test run.
          Hide
          northshorefiend James Alan Shepherd added a comment - - edited

          Me too.

          I have a 'database create' script that un-surprisingly creates tables and populates them.

          Running on 10.3.1.4 I get an

          ERROR XSAI2: The conglomerate (-15) requested does not exist.

          Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source)
          at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
          at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)

          Running on 10.2.2.0 everything is fine.

          Spooky.

          The environment is just using the corresponding version of the embedded driver with all the jars on the classpath, starting with an empty derby.home directory (except for derby.properties turning up logging).

          There's a few posts on the net about this error having something to do with corrupted files. Clearly this doesn't apply in this case, except that 10.3.1.4 may be having it's own nervous breakdown.

          100% repeatable. Would I get anything by connecting a debugger? With all this (Unknown Source) I doubt it. I don't rate my chances of getting a good test case quickly.

          oh and:

          ~ $ java -version; uname -primo
          java version "1.6.0_03"
          Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
          Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode)
          2.6.23-gentoo-r3-6 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux

          For the sake of it I tested using:

          java version "1.6.0_03"
          Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
          Java HotSpot(TM) Server VM (build 1.6.0_03-b05, mixed mode)

          with 10.3.1.4 and still get the above error.

          Show
          northshorefiend James Alan Shepherd added a comment - - edited Me too. I have a 'database create' script that un-surprisingly creates tables and populates them. Running on 10.3.1.4 I get an ERROR XSAI2: The conglomerate (-15) requested does not exist. Caused by: ERROR XSAI2: The conglomerate (-15) requested does not exist. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown Source) at org.apache.derby.impl.store.access.RAMTransaction.openScan(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.TemporaryRowHolderResultSet.getNextRow(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) Running on 10.2.2.0 everything is fine. Spooky. The environment is just using the corresponding version of the embedded driver with all the jars on the classpath, starting with an empty derby.home directory (except for derby.properties turning up logging). There's a few posts on the net about this error having something to do with corrupted files. Clearly this doesn't apply in this case, except that 10.3.1.4 may be having it's own nervous breakdown. 100% repeatable. Would I get anything by connecting a debugger? With all this (Unknown Source) I doubt it. I don't rate my chances of getting a good test case quickly. oh and: ~ $ java -version; uname -primo java version "1.6.0_03" Java(TM) SE Runtime Environment (build 1.6.0_03-b05) Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) 2.6.23-gentoo-r3-6 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux For the sake of it I tested using: java version "1.6.0_03" Java(TM) SE Runtime Environment (build 1.6.0_03-b05) Java HotSpot(TM) Server VM (build 1.6.0_03-b05, mixed mode) with 10.3.1.4 and still get the above error.
          Hide
          knutanders Knut Anders Hatlen added a comment -

          Is the problem reproducible, and if so, is it possible to post code that demonstrates the problem or a step-by-step description of how to reproduce it?

          Show
          knutanders Knut Anders Hatlen added a comment - Is the problem reproducible, and if so, is it possible to post code that demonstrates the problem or a step-by-step description of how to reproduce it?

            People

            • Assignee:
              dyret Dyre Tjeldvoll
              Reporter:
              hallorant Tim Halloran
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development