Triaged for 10.5.2: Assigned normal urgency and noted that repro is available.

From a small experiment running the attached repro, where I made OverflowInputStream throw an exception if it detects that the specified overflow page cannot be found, the following lock dumps shows that an extra lock is set when using the client driver:

• • Embedded:
    /---- Lock dump 1
    477 | TABLE | IS | BLOBTAB | Tablelock | GRANT | T | 1 | null |
    Lock dump 1 -----/
    Caught IOException:EOF reached prematurely
    0 rows in BLOBCLOB

• • Client:
    /---- Lock dump 1
    368 | TABLE | IS | BLOBTAB | Tablelock | GRANT | T | 2 | null |
    368 | ROW | S | BLOBTAB | (2,6) | GRANT | T | 1 | null |
    Lock dump 1 -----/
    // Results in lock timeout.

Haven't investigated further, but the repro code is identical except for the connection URL so there is definitely a difference between embedded and client/server.
Also, are there any valid situations where OverflowInputStream.fillByteHolder should fail to get the overflow page when an overflow page has been specified?
(the next overflow page is set by a callback, according to the comments)

Finally, if the suggested fix is pursued, we might want to beef up the error message. Any suggestions?

Attached patch 1a, which fixes the problem as outlined in the comment above. I also incorporated the repro/test Kathey has written.
Should we try to say something about what went wrong in the error message, such that the user has a chance to understand what went wrong?

Patch ready for review.

Regression tests passed.

The approach in the patch looks fine to me. I think you're right that the error message is somewhat too developer-centric, and it may be difficult for users to understand what the problem is. What about this instead: "Could not read further because another transaction has modified the value." ?

Thanks for looking at the patch, Knut Anders.

Your proposed message sound better to me, and I think it doesn't lie That is, I believe there aren't any other valid reasons causing that exception to be thrown. If you use a stricter isolation level I think locks will be put in place to avoid that a different transaction deletes the value you are currently accessing.
The only other reason I can think of, is if we have a bug in the store specifying the wrong overflow page id.

I'll change the error message and commit the patch shortly.

Attached patch 1b and committed it to trunk with revision 908586.

Back-porting this fix requires some manual work, due to conflicts.
Does anyone feel strong about back-porting the fix?

I haven't received any feedback on the back-porting issue.
Resolving issue, ready for verification.

I am looking at backporting this to the 10.5 branch.

backporting change #908586 from trunk to 10.5 branch.

m105_jdk16:178>svn commit
Sending .
Sending java/engine/org/apache/derby/impl/store/raw/data/OverflowInputStream.java
Sending java/engine/org/apache/derby/loc/messages.xml
Sending java/shared/org/apache/derby/shared/common/reference/MessageId.java
Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/BLOBTest.java
Transmitting file data ....

backported to 10.5. resolving as fixed and resetting original owner.