Derby
  1. Derby
  2. DERBY-4455

Prepared statement failure with CLOB: Stream has already been read and end-of-file reached and cannot be re-used.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.5.3.0
    • Fix Version/s: 10.5.3.1, 10.6.1.0
    • Component/s: Network Server
    • Labels:
      None
    • Environment:
      Mac OS X 10.6.2, Java 6, Bitronix JTA
    • Bug behavior facts:
      Crash, Regression

      Description

      Possibly related to #4332?

      We have encountered an error when using Prepared Statements and CLOBs. I have read:

      http://db.apache.org/derby/papers/JDBCImplementation.html#setAsciiStream%2CsetBinaryStream%2CsetCharacterStream

      But it does not seem applicable, as we are not re-using a stream.

      The environment is this:

      1. Java 6
      2. Derby 10.5.3.0
      3. Bitronix JTA 1.3.3

      We're actually using Hibernate, but I eliminated it from the equation (and the problem persists).

      A summary of the failure flow is this:

      1. Start a transaction
      2. Obtain a connection from a pool of connections (for this test, the pool size is pinned at 1)
      3. Prepare a statement that inserts a CLOB.
      4. Set the parameters
      5. Add the prepared statement to a batch (but we only batch 1 – this is to emulate what hibernate is doing as closely as possible).
      6. Execute the batch.

      Everything up to this point works.

      7. Repeat steps 1-6. But this time, the connection will be reused from the pool, and the statement will be gotten from a prepared statement cache (maintained by bitronix). I.e. the prepared statement is re-used.
      8. Observe the following failure:

      org.apache.derby.client.am.BatchUpdateException: Non-atomic batch failure. The batch was submitted, but at least one exception occurred on an individual member of the batch. Use getNextException() to retrieve the exceptions for specific batched elements.
      at org.apache.derby.client.am.Agent.endBatchedReadChain(Unknown Source)
      at org.apache.derby.client.am.PreparedStatement.executeBatchRequestX(Unknown Source)
      at org.apache.derby.client.am.PreparedStatement.executeBatchX(Unknown Source)
      at org.apache.derby.client.am.PreparedStatement.executeBatch(Unknown Source)
      at bitronix.tm.resource.jdbc.JdbcPreparedStatementHandle.executeBatch(JdbcPreparedStatementHandle.java:248)
      at org.dancernetworks.TestFailure.doInsert(TestFailure.java:134)
      at org.dancernetworks.TestFailure.doPrepared(TestFailure.java:110)
      at org.dancernetworks.TestFailure.main(TestFailure.java:55)
      Nov 30, 2009 10:29:31 PM bitronix.tm.BitronixTransactionManager shutdown
      INFO: shutting down Bitronix Transaction Manager
      An IOException was thrown when reading a 'java.sql.String' from an InputStream.
      java.sql.SQLException: An IOException was thrown when reading a 'java.sql.String' from an InputStream.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedResultSet.noStateChangeException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement.transferParameters(Unknown Source)
      at org.apache.derby.jdbc.XAStatementControl.getRealPreparedStatement(Unknown Source)
      at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.getPreparedStatement(Unknown Source)
      at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.getStatement(Unknown Source)
      at org.apache.derby.iapi.jdbc.BrokeredStatement.close(Unknown Source)
      at org.apache.derby.impl.drda.DRDAStatement.close(Unknown Source)
      at org.apache.derby.impl.drda.Database.close(Unknown Source)
      at org.apache.derby.impl.drda.Session.close(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.closeSession(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
      Caused by: java.sql.SQLException: An IOException was thrown when reading a 'java.sql.String' from an InputStream.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      ... 15 more
      Caused by: java.sql.SQLException: Java exception: 'Stream has already been read and end-of-file reached and cannot be re-used.: java.io.EOFException'.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      ... 12 more
      Caused by: java.io.EOFException: Stream has already been read and end-of-file reached and cannot be re-used.
      at org.apache.derby.iapi.types.ReaderToUTF8Stream.read(Unknown Source)
      at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:320)
      at org.apache.derby.iapi.types.SQLChar.readExternal(Unknown Source)
      at org.apache.derby.iapi.types.SQLChar.getString(Unknown Source)
      at org.apache.derby.iapi.types.SQLChar.setFrom(Unknown Source)
      at org.apache.derby.iapi.types.DataType.setValue(Unknown Source)
      at org.apache.derby.impl.sql.GenericParameterValueSet.transferDataValues(Unknown Source)
      at org.apache.derby.impl.sql.execute.BaseActivation.setParameters(Unknown Source)
      at org.apache.derby.impl.sql.GenericActivationHolder.setParameters(Unknown Source)
      ... 10 more

      Attached is an archived Eclipse project of a self-contained reproduction. It includes everything needed to run, including the Bitronix 1.3.3 jar.

      1. ReproAttemptDerby4455.java
        2 kB
        Kathey Marsden
      2. derby-4455-1c.diff
        3 kB
        Kristian Waagan
      3. derby-4455-1b.diff
        3 kB
        Kristian Waagan
      4. derby-4455-1a.diff
        3 kB
        Kristian Waagan
      5. DerbyFailure.zip
        3.24 MB
        Brett Wooldridge

        Issue Links

          Activity

          Hide
          Brett Wooldridge added a comment -

          Zipped eclipse java project with failure reproduction.

          Show
          Brett Wooldridge added a comment - Zipped eclipse java project with failure reproduction.
          Hide
          Kristian Waagan added a comment -

          Patch 1a is one way of fixing the problem.
          You still can't reuse streams, but this should address the issue you're seeing in the repro.
          I haven't verified if this happens only in XA environments, or if some kind of reuse / pooling is enough to trigger the bug.

          I've started the regression test suite, I'll post the results tomorrow.

          Show
          Kristian Waagan added a comment - Patch 1a is one way of fixing the problem. You still can't reuse streams, but this should address the issue you're seeing in the repro. I haven't verified if this happens only in XA environments, or if some kind of reuse / pooling is enough to trigger the bug. I've started the regression test suite, I'll post the results tomorrow.
          Hide
          Kristian Waagan added a comment -

          Patch available for testing / review.

          Show
          Kristian Waagan added a comment - Patch available for testing / review.
          Hide
          Knut Anders Hatlen added a comment -

          Kristian, could you add some details about what's causing the bug? I didn't quite get that by reading the comments and the patch.

          The server-side stack trace appears to happen when the session is being closed. Do we really need to read the parameter values in order to close the statement?

          Show
          Knut Anders Hatlen added a comment - Kristian, could you add some details about what's causing the bug? I didn't quite get that by reading the comments and the patch. The server-side stack trace appears to happen when the session is being closed. Do we really need to read the parameter values in order to close the statement?
          Hide
          Brett Wooldridge added a comment -

          Kristian, the patch works for me. I don't particularly care about re-using streams. Because we just found this issue when we enabled the PreparedStatement cache, I'll let it run for a day or two to see if any other issues are uncovered.

          Thanks.

          Show
          Brett Wooldridge added a comment - Kristian, the patch works for me. I don't particularly care about re-using streams. Because we just found this issue when we enabled the PreparedStatement cache, I'll let it run for a day or two to see if any other issues are uncovered. Thanks.
          Hide
          Kristian Waagan added a comment -

          Uploading a revised patch 1b, with some changed comments and some trivial changes to the code. It should be functionally equal to patch 1a.

          Knut> The server-side stack trace appears to happen when the session is being closed.

          Yes, it appears to, but I don't think it is any more. Printing a stack trace at GenericParameterValueSet.transferDataValue gives me:
          at org.apache.derby.impl.sql.GenericParameterValueSet.transferDataValues(GenericParameterValueSet.java:249)
          at org.apache.derby.impl.sql.execute.BaseActivation.setParameters(BaseActivation.java:1300)
          at org.apache.derby.impl.sql.GenericActivationHolder.setParameters(GenericActivationHolder.java:246)
          at org.apache.derby.impl.jdbc.EmbedPreparedStatement.transferParameters(EmbedPreparedStatement.java:1662)
          at org.apache.derby.jdbc.XAStatementControl.getRealPreparedStatement(XAStatementControl.java:169)
          at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.getPreparedStatement(BrokeredPreparedStatement.java:531)
          at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.setInt(BrokeredPreparedStatement.java:160)
          at org.apache.derby.impl.drda.DRDAConnThread.readAndSetParams(DRDAConnThread.java:4578)
          at org.apache.derby.impl.drda.DRDAConnThread.parseSQLDTA_work(DRDAConnThread.java:4507)
          at org.apache.derby.impl.drda.DRDAConnThread.parseSQLDTA(DRDAConnThread.java:4379)
          at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(DRDAConnThread.java:4254)
          at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(DRDAConnThread.java:4084)
          at org.apache.derby.impl.drda.DRDAConnThread.processCommands(DRDAConnThread.java:1003)
          at org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:290)

          Further investigation points to DERBY-4310, DERBY-4155 and DERBY-4334, where a commit for the former two changed the code closing a statement. The stack trace you refer to must be from an earlier release. As a side note, I'm not seeing the exception printed to derby.log, even with derby.error.stream.logSeverityLevel=0.
          My understanding of XA isn't as good as it should have been, but running the repro with the embedded driver (using EmbeddedXADataSource) doesn't trigger the bug. My question then is whether the "connection switching" behavior (see XAStatementControl.getRealPreparedStatement) seen with the client driver is a necessity or if the implementation is suboptimal.
          Out of curiosity I also enabled the Derby client side JDBC statement cache for XA (requires some code changes) and disabled the one in Bitronix. I observed the same behavior, including the bug.

          As for the issue addressed by the patch, what happens is that Derby detects that the application connection has changed when it is about to execute the prepared statement. It then has to create a new prepared statement on the current connection. After preparing a new statement, it transfers the arguments that were set on the old statement to the new one. This was done in a way that would materialize values represented as a stream. In this specific case, the stream had already been drained and an EOFException was thrown when trying to read it again.

          I believe the same bug can be triggered by using only the embedded driver, but it requires a different repro (switching between global and local XA transactions or something?).

          Finally, the regression test run with patch 1a was clean.

          Show
          Kristian Waagan added a comment - Uploading a revised patch 1b, with some changed comments and some trivial changes to the code. It should be functionally equal to patch 1a. Knut> The server-side stack trace appears to happen when the session is being closed. Yes, it appears to, but I don't think it is any more. Printing a stack trace at GenericParameterValueSet.transferDataValue gives me: at org.apache.derby.impl.sql.GenericParameterValueSet.transferDataValues(GenericParameterValueSet.java:249) at org.apache.derby.impl.sql.execute.BaseActivation.setParameters(BaseActivation.java:1300) at org.apache.derby.impl.sql.GenericActivationHolder.setParameters(GenericActivationHolder.java:246) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.transferParameters(EmbedPreparedStatement.java:1662) at org.apache.derby.jdbc.XAStatementControl.getRealPreparedStatement(XAStatementControl.java:169) at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.getPreparedStatement(BrokeredPreparedStatement.java:531) at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.setInt(BrokeredPreparedStatement.java:160) at org.apache.derby.impl.drda.DRDAConnThread.readAndSetParams(DRDAConnThread.java:4578) at org.apache.derby.impl.drda.DRDAConnThread.parseSQLDTA_work(DRDAConnThread.java:4507) at org.apache.derby.impl.drda.DRDAConnThread.parseSQLDTA(DRDAConnThread.java:4379) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(DRDAConnThread.java:4254) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(DRDAConnThread.java:4084) at org.apache.derby.impl.drda.DRDAConnThread.processCommands(DRDAConnThread.java:1003) at org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:290) Further investigation points to DERBY-4310 , DERBY-4155 and DERBY-4334 , where a commit for the former two changed the code closing a statement. The stack trace you refer to must be from an earlier release. As a side note, I'm not seeing the exception printed to derby.log, even with derby.error.stream.logSeverityLevel=0. My understanding of XA isn't as good as it should have been, but running the repro with the embedded driver (using EmbeddedXADataSource) doesn't trigger the bug. My question then is whether the "connection switching" behavior (see XAStatementControl.getRealPreparedStatement) seen with the client driver is a necessity or if the implementation is suboptimal. Out of curiosity I also enabled the Derby client side JDBC statement cache for XA (requires some code changes) and disabled the one in Bitronix. I observed the same behavior, including the bug. As for the issue addressed by the patch, what happens is that Derby detects that the application connection has changed when it is about to execute the prepared statement. It then has to create a new prepared statement on the current connection. After preparing a new statement, it transfers the arguments that were set on the old statement to the new one. This was done in a way that would materialize values represented as a stream. In this specific case, the stream had already been drained and an EOFException was thrown when trying to read it again. I believe the same bug can be triggered by using only the embedded driver, but it requires a different repro (switching between global and local XA transactions or something?). Finally, the regression test run with patch 1a was clean.
          Hide
          Brett Wooldridge added a comment -

          Kristian, if I could, I'd like to ask a few questions about your comments. You said, " what happens is that Derby detects that the application connection has changed when it is about to execute the prepared statement", what exactly changed from Derby's perspective? Then you said, "After preparing a new statement, it transfers the arguments that were set on the old statement to the new one" Does this involve a "real" preparation of a statement? And therefore does this behavior defeat the purpose of a PreparedStatement cache (either external or internal)?

          With respect to the first question, is the "change" in the application connection that is detected by Derby a result of XA management? Given that the statement cache is per-connection, and that the connection is re-used (in this case), why does Derby require a "re-prepare" and copy of arguments?

          Thanks.

          Show
          Brett Wooldridge added a comment - Kristian, if I could, I'd like to ask a few questions about your comments. You said, " what happens is that Derby detects that the application connection has changed when it is about to execute the prepared statement", what exactly changed from Derby's perspective? Then you said, "After preparing a new statement, it transfers the arguments that were set on the old statement to the new one" Does this involve a "real" preparation of a statement? And therefore does this behavior defeat the purpose of a PreparedStatement cache (either external or internal)? With respect to the first question, is the "change" in the application connection that is detected by Derby a result of XA management? Given that the statement cache is per-connection, and that the connection is re-used (in this case), why does Derby require a "re-prepare" and copy of arguments? Thanks.
          Hide
          Kristian Waagan added a comment -

          Hi Brett,

          Answering your questions requires some more investigation, and I don't have the cycles right now. I'll try to get back to them later.

          I can give some information right away though.
          Brett> ... [a] Does this involve a "real" preparation of a statement? [b] And therefore does this behavior defeat the purpose of a PreparedStatement cache (either external or internal)?
          a) In a way it does on the server side, yes. However, the server has a statement cache of its own, across connections.
          b) Depends on what you mean. It helps a lot when using the client driver, because you don't have to go across the network as much. My experience comes from using a ClientConnectionPoolDataSource with the internal cache. I don't know how the internal cache compares to an external cache in terms of performance.
          On the server side, you still benefit from the cache (on the client side), because the "connection change" I mentioned doesn't happen for every operation (I'll need to investigate to figure out what causes this perceived change of state). If the server has to re-prepare the statement all the time, it will put unnecessary strain on the server side statement cache - and it may become a bottleneck. I don't remember having seen this as a problem earlier, but then I haven't been running benchmarks in an XA environment. We now also have session state piggy-backing (schema name and isolation level), which was required to implement the client side JDBC prepared statement cache.

          Brett> ... Given that the statement cache is per-connection, and that the connection is re-used (in this case), why does Derby require a "re-prepare" and copy of arguments?
          I can't answer this properly, but my theory is that the embedded connection on the server side changes. Again, I'll have to confirm this, and figure out why it happens if the theory is correct. There might be an XA specific reason for this, some kind of mismatch between the client and embedded connection, or perhaps due to a side-effect of some internal mechanism.

          Two questions for you:
          1) Have you explicitly turned on connection validation using VALUES 1, or is this something Bitronix does by default?
          2) Are you having performance issues with your application?

          And thanks for providing the runnable repro and for taking the patch for a spin.

          Show
          Kristian Waagan added a comment - Hi Brett, Answering your questions requires some more investigation, and I don't have the cycles right now. I'll try to get back to them later. I can give some information right away though. Brett> ... [a] Does this involve a "real" preparation of a statement? [b] And therefore does this behavior defeat the purpose of a PreparedStatement cache (either external or internal)? a) In a way it does on the server side, yes. However, the server has a statement cache of its own, across connections. b) Depends on what you mean. It helps a lot when using the client driver, because you don't have to go across the network as much. My experience comes from using a ClientConnectionPoolDataSource with the internal cache. I don't know how the internal cache compares to an external cache in terms of performance. On the server side, you still benefit from the cache (on the client side), because the "connection change" I mentioned doesn't happen for every operation (I'll need to investigate to figure out what causes this perceived change of state). If the server has to re-prepare the statement all the time, it will put unnecessary strain on the server side statement cache - and it may become a bottleneck. I don't remember having seen this as a problem earlier, but then I haven't been running benchmarks in an XA environment. We now also have session state piggy-backing (schema name and isolation level), which was required to implement the client side JDBC prepared statement cache. Brett> ... Given that the statement cache is per-connection, and that the connection is re-used (in this case), why does Derby require a "re-prepare" and copy of arguments? I can't answer this properly, but my theory is that the embedded connection on the server side changes. Again, I'll have to confirm this, and figure out why it happens if the theory is correct. There might be an XA specific reason for this, some kind of mismatch between the client and embedded connection, or perhaps due to a side-effect of some internal mechanism. Two questions for you: 1) Have you explicitly turned on connection validation using VALUES 1, or is this something Bitronix does by default? 2) Are you having performance issues with your application? And thanks for providing the runnable repro and for taking the patch for a spin.
          Hide
          Brett Wooldridge added a comment -

          Q1) Have you explicitly turned on connection validation using VALUES 1, or is this something Bitronix does by default?
          I set the test query in the config, but I think it was recommended by bitronix (or some other connection pool doc).

          Q2) Are you having performance issues with your application?
          Not particularly, just trying to optimize things a bit. SQL was showing up high during some profiling sessions. But the SQL can't particularly be optimized – it's just straight inserts. Now batched. In the past, my experience with MySQL and SQL Server was that re-using prepared statements made a huge difference – like 10-20x in some cases. I actually wrote the prepared statement cache in the jTDS (open source SQL Server) driver as a result of work at that job (probably 5 years ago now).

          No worries on runnable repro, I know it can cut the diagnostic time significantly.

          Show
          Brett Wooldridge added a comment - Q1) Have you explicitly turned on connection validation using VALUES 1, or is this something Bitronix does by default? I set the test query in the config, but I think it was recommended by bitronix (or some other connection pool doc). Q2) Are you having performance issues with your application? Not particularly, just trying to optimize things a bit. SQL was showing up high during some profiling sessions. But the SQL can't particularly be optimized – it's just straight inserts. Now batched. In the past, my experience with MySQL and SQL Server was that re-using prepared statements made a huge difference – like 10-20x in some cases. I actually wrote the prepared statement cache in the jTDS (open source SQL Server) driver as a result of work at that job (probably 5 years ago now). No worries on runnable repro, I know it can cut the diagnostic time significantly.
          Hide
          Brett Wooldridge added a comment -

          I consider this bug fixed by the provided patch. Therefore you are free to close the issue.

          However, if there is indeed unnecessary server-side "re-preparation" of the statements (see org.apache.derby.impl.jdbc.EmbedPreparedStatement.transferParameters in the stack trace) in the case of XA connections, I would like a separate issue to be opened. Other than that efficiency concern, the actual defect appears to be fixed by the patch and I have not encountered this exception since applying the patch.

          I would like to see this fix integrated into the next release because I prefer not to run non-released versions of jars.

          Show
          Brett Wooldridge added a comment - I consider this bug fixed by the provided patch. Therefore you are free to close the issue. However, if there is indeed unnecessary server-side "re-preparation" of the statements (see org.apache.derby.impl.jdbc.EmbedPreparedStatement.transferParameters in the stack trace) in the case of XA connections, I would like a separate issue to be opened. Other than that efficiency concern, the actual defect appears to be fixed by the patch and I have not encountered this exception since applying the patch. I would like to see this fix integrated into the next release because I prefer not to run non-released versions of jars.
          Hide
          Knut Anders Hatlen added a comment -

          One comment to the patch:

          + // NOTE: The length is ignored (use a non-sense value, it
          + // hopefully fails if required in the future).
          + pvstarget.getParameterForSet.setValue(is, -3);

          The length is only ignored by SQLChar.setValue() (and, by inheritance, SQLClob). SQLBinary.setValue() uses the length argument, as far as I can see. It may be more robust to create a new setValue() method with no length argument and with well-defined behaviour.

          Show
          Knut Anders Hatlen added a comment - One comment to the patch: + // NOTE: The length is ignored (use a non-sense value, it + // hopefully fails if required in the future). + pvstarget.getParameterForSet .setValue(is, -3); The length is only ignored by SQLChar.setValue() (and, by inheritance, SQLClob). SQLBinary.setValue() uses the length argument, as far as I can see. It may be more robust to create a new setValue() method with no length argument and with well-defined behaviour.
          Hide
          Kristian Waagan added a comment -

          Knut Anders> ... It may be more robust to create a new setValue() method with no length argument and with well-defined behaviour.

          I agree adding another value would be more robust.
          Looking at the existing code, it seems -1 is already used to indicate that the length is unknown. I found a total of 7 uses of the existing method, so it isn't heavily used.
          Would it make sense to improve the documentation of the existing method (in DataValueDescriptor) and verifying the current use, instead of adding yet another method?

          After agreeing on this issue, I plan to commit the patch to trunk and backport it to the 10.5 branch. I might also merge it with 10.4 if it merges cleanly.
          I think it is an incremental improvement, and it would be nice to follow up on the XA behavior later.

          Show
          Kristian Waagan added a comment - Knut Anders> ... It may be more robust to create a new setValue() method with no length argument and with well-defined behaviour. I agree adding another value would be more robust. Looking at the existing code, it seems -1 is already used to indicate that the length is unknown. I found a total of 7 uses of the existing method, so it isn't heavily used. Would it make sense to improve the documentation of the existing method (in DataValueDescriptor) and verifying the current use, instead of adding yet another method? After agreeing on this issue, I plan to commit the patch to trunk and backport it to the 10.5 branch. I might also merge it with 10.4 if it merges cleanly. I think it is an incremental improvement, and it would be nice to follow up on the XA behavior later.
          Hide
          Knut Anders Hatlen added a comment -

          Sounds reasonable, although I don't quite understand how the length argument is used. Could this somehow make the length parameter passed to one of the top-level JDBC streaming methods getting lost?

          Show
          Knut Anders Hatlen added a comment - Sounds reasonable, although I don't quite understand how the length argument is used. Could this somehow make the length parameter passed to one of the top-level JDBC streaming methods getting lost?
          Hide
          Kristian Waagan added a comment -

          I logged DERBY-4515 to clarify the usage of the length argument (a patch is awaiting review).
          Regarding your concern about the top-level JDBC length parameter(s) getting lost, I think setValue(InputStream,int) is a way we can pass the information on. This is currently done for binary types, but not for the character types. In the latter case, there is another mechanism (CharacterStreamDescriptor) used to achieve the same effect - at least for Clobs where it may really matter.

          After DERBY-4515 has been resolved, a small change has to be made to the patch attached this issue. I will then commit it and backport it to 10.5.

          Show
          Kristian Waagan added a comment - I logged DERBY-4515 to clarify the usage of the length argument (a patch is awaiting review). Regarding your concern about the top-level JDBC length parameter(s) getting lost, I think setValue(InputStream,int) is a way we can pass the information on. This is currently done for binary types, but not for the character types. In the latter case, there is another mechanism (CharacterStreamDescriptor) used to achieve the same effect - at least for Clobs where it may really matter. After DERBY-4515 has been resolved, a small change has to be made to the patch attached this issue. I will then commit it and backport it to 10.5.
          Hide
          Kristian Waagan added a comment -

          Attached patch 1c, adjusted to use the newly introduced constant DataValueDescriptor.UNKNOWN_LOGICAL_LENGTH when calling DVD.setStream.

          Committed 1c to trunk with revision 901760 and back-ported to the 10.5 branch with revision 901761.
          I will mark this issue as resolved when the automated tests pass.

          Show
          Kristian Waagan added a comment - Attached patch 1c, adjusted to use the newly introduced constant DataValueDescriptor.UNKNOWN_LOGICAL_LENGTH when calling DVD.setStream. Committed 1c to trunk with revision 901760 and back-ported to the 10.5 branch with revision 901761. I will mark this issue as resolved when the automated tests pass.
          Hide
          Kristian Waagan added a comment -

          No problems reported by the nightly test runs.
          Resolving issue.

          Brett, feel free to close this issue if you think it has been addressed.

          Show
          Kristian Waagan added a comment - No problems reported by the nightly test runs. Resolving issue. Brett, feel free to close this issue if you think it has been addressed.
          Hide
          Brett Wooldridge added a comment -

          Closed, fixed.

          Show
          Brett Wooldridge added a comment - Closed, fixed.
          Hide
          Kathey Marsden added a comment -

          Was this issue a regression? I am working with a user that is seeing something similar after upgrading to 10.5 from 10.3, but they don't have this fix yet. I am hoping giving them the patch will fix their problem as it sounds so similar, but don't actually see this marked as a regression from 10.3

          Show
          Kathey Marsden added a comment - Was this issue a regression? I am working with a user that is seeing something similar after upgrading to 10.5 from 10.3, but they don't have this fix yet. I am hoping giving them the patch will fix their problem as it sounds so similar, but don't actually see this marked as a regression from 10.3
          Hide
          Kristian Waagan added a comment -

          I don't know if this issue was a regression.

          I don't have the cycles right now, but finding out should be simple enough (fingers crossed...):

          • download and unpack the repro
          • replace the Derby jars with older versions
            (- delete existing database?)
          • run the repro

          The details are a bit murky for me, but I think running the repro is pretty simple.

          Show
          Kristian Waagan added a comment - I don't know if this issue was a regression. I don't have the cycles right now, but finding out should be simple enough (fingers crossed...): download and unpack the repro replace the Derby jars with older versions (- delete existing database?) run the repro The details are a bit murky for me, but I think running the repro is pretty simple.
          Hide
          Kathey Marsden added a comment -

          Thanks Kristian for responding so quickly. I will check it out and mark the box accordingly. just wanted to check and see anyone knew off hand.

          Show
          Kathey Marsden added a comment - Thanks Kristian for responding so quickly. I will check it out and mark the box accordingly. just wanted to check and see anyone knew off hand.
          Hide
          Kathey Marsden added a comment -

          I wanted to report that this was indeed the fix needed for the user that reported a regression from 10.3. Thanks Kristian and Brett!

          I am trying to make a stand-alone reproduction/regression test to check in but am having some trouble popping the bug with 10.5.3.0 - (802917), before the fix. Attached is my ReproAttemptDerby4455.java.

          I'll keep working on it. It would be good to have a regression test for this issue.

          Show
          Kathey Marsden added a comment - I wanted to report that this was indeed the fix needed for the user that reported a regression from 10.3. Thanks Kristian and Brett! I am trying to make a stand-alone reproduction/regression test to check in but am having some trouble popping the bug with 10.5.3.0 - (802917), before the fix. Attached is my ReproAttemptDerby4455.java. I'll keep working on it. It would be good to have a regression test for this issue.
          Hide
          Kathey Marsden added a comment -

          Marking as a regression as it was reported on upgrade to 10.5 from 10.3 with the same application.

          Show
          Kathey Marsden added a comment - Marking as a regression as it was reported on upgrade to 10.5 from 10.3 with the same application.

            People

            • Assignee:
              Kristian Waagan
              Reporter:
              Brett Wooldridge
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development