State of InputStream retrieved from resultset is not clean , if there exists previous InputStream .

• Key: DERBY-721
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Tomohito Nakayama
• Reporter: Tomohito Nakayama
• Component/s: JDBC, Network Client
• Fix Version/s: 10.2.1.6

Environment

naka@rufelza:~/derby/dev/trunk$ cat /proc/version
Linux version 2.6.12-1-386 ([e-mail]) (gcc version 4.0.2 20050917 (prerelease) (Debian 4.0.1-8)) #1 Tue Sep 27 12:41:08 JST 2005
naka@rufelza:~/derby/dev/trunk$ java -version
java version "1.4.2_10"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_10-b03)
Java HotSpot(TM) Client VM (build 1.4.2_10-b03, mixed mode)

Description

State of InputStream retrieved from ResultSet was not clean , if there exists previous InputStream retrieved from ResultSet .

Test code ...
PreparedStatement pst = conn.prepareStatement("insert into a(b) values(?)");

byte[] data = new byte[1024 * 1024];
for(int i = 0;
i < 1024 * 1024;
i ++){
data[i] = (byte)(i % 256);
}

pst.setBinaryStream(1,new ByteArrayInputStream(data),data.length);
pst.executeUpdate();
pst.close();


st = conn.createStatement();
ResultSet rs = st.executeQuery("select b from a");
rs.next();

InputStream is = rs.getBinaryStream(1);
System.out.println("Here goes first stream");
System.out.println(is.read());
System.out.println(is.read());
System.out.println(is.read());

is = rs.getBinaryStream(1);
System.out.println("Here goes 2nd stream");
System.out.println(is.read());
System.out.println(is.read());
System.out.println(is.read());

Result ....
naka@rufelza:~/derby/test/20051121$ java testLob
Here goes first stream
0
1
2
Here goes 2nd stream
7
8
9

It is expected that result printed from first stream is as same as result printed from 2nd.

Comments

These are test program for reproduction of the phenomena what I found , and result of the program .

[Tomohito Nakayama]

I found that this phenomena does not happen if data length was small, such as 32.

Class of inputstream was different between when phenomena happen and when does not.

When phenomena happens:
main[1] eval is
 is = "org.apache.derby.impl.jdbc.BinaryToRawStream@29d294"

When phenomena does not happen.
main[1] eval is
 is = "org.apache.derby.iapi.services.io.NewByteArrayInputStream@19abd2b"

I suspect closing process of BinaryToRawStream ...

[Tomohito Nakayama]

I found that BinaryToRawStream is subclass of java.io.FilteredInputStream and
would just close source InputStream when close method was called.

However, source InputStream was shared between bunch of destination InputStream.
Then, just closing source InputStream may be not correct.

I found when this phenomena was found , substantial class of source InputStream object implements Resetable .

I will call method to reset source InputStream object ,if it was Resetable object , in BinaryToRawSteream.

[Tomohito Nakayama]

Description of the patch:
  Modified close method of org.apache.derby.impl.jdbc.BinaryToRawStream to reset source InputStream which implements Resetable interface,
  because the source InputStream is shared with following InputStream.

Test:
 execute derbyall and found no error.

[Tomohito Nakayama]

Committed

Sending java/engine/org/apache/derby/impl/jdbc/BinaryToRawStream.java
Sending java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/resultsetStream.out
Sending java/testing/org/apache/derbyTesting/functionTests/master/resultsetStream.out
Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/resultsetStream.java
Transmitting file data ....
Committed revision 349079.

[Tomohito Nakayama]

I found same problem in getCharacterStream ...

[Tomohito Nakayama]

These are test to reproduce phenomena found at getCharacterStream

[Tomohito Nakayama]

Patch for the phenomena found in getCharacterStream

Description:
   Modified to reset the source InputStream instead of closing it , when org.apache.derby.impl.jdbc.UTF8Reader was closed.

Test:
   Executed derbyall and does not found new error other than found in http://www.multinet.no/~solberg/public/Apache/Derby/Limited/index.html already.

[Tomohito Nakayama]

patch commited.


Sending java/engine/org/apache/derby/impl/jdbc/EmbedClob.java
Sending java/engine/org/apache/derby/impl/jdbc/UTF8Reader.java
Sending java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/resultsetStream.out
Sending java/testing/org/apache/derbyTesting/functionTests/master/resultsetStream.out
Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/resultsetStream.java
Transmitting file data .....
Committed revision 349718.

[Tomohito Nakayama]

Dan wrote:

The error handling is incorrect here, and I think in at least one other
place in the patch.

1) Derby should not be printing stack traces to System.err, it should
just throw an exception. It's up to the calling application to print the
stack trace, if it wants.

2) There are utility methods, e.g. noStateChangeLOB, to create new
SQLExceptions, creating an exception like this will not include a
SQLState and will not link the original IOException to the SQLException.

Thanks,
Dan.

[Tomohito Nakayama]

Description of patch :
   I judged that SQLException should not be thrown in InputStream#close/Reader#close.
   I remove calling resetStream method from close method and
     implemented Resetable interface in UTF8Reader and BinaryToRawStream and
     reset them from EmbedResultSet.

Test of patch:
   I executed derbyall and does not found new error.

[Tomohito Nakayama]

http://mail-archives.apache.org/mod_mbox/db-derby-dev/200512.mbox/[e-mail]%3e

As discussed in series of mail refered by URL above ,
it should be restricted to get InputStream for same value in result row twice from a ResultSet .

[Tomohito Nakayama]

Description of this patch :
  This patch rollback DERBY-721.patch and DERBY-721_2.patch .

Test :
  Executed derbyall and found no error .

[Tomohito Nakayama]

I commited DERBY-721_rollback_1+2.patch .

Sending java/engine/org/apache/derby/impl/jdbc/BinaryToRawStream.java
Sending java/engine/org/apache/derby/impl/jdbc/EmbedClob.java
Sending java/engine/org/apache/derby/impl/jdbc/UTF8Reader.java
Sending java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/resultsetStream.out
Sending java/testing/org/apache/derbyTesting/functionTests/master/resultsetStream.out
Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/resultsetStream.java
Transmitting file data ......
Committed revision 355437.

[Tomohito Nakayama]

Description of patch :
 What is done:
  1: Throw newly defined exception when stream for same column in result was retrieved from a resultset , both in embed and network client.
  2: Add new test code for 1.

 What is *not* done:
  I think these should be considered in other patch/issue.
  1:Exception was *not* thrown when stream for same column in result was retrieved from Blob/Clob .
  2:Exception was *not* thrown when value , other than stream , was retrieved from result set ,after stream for the same column was retrieved.


Test:
 Executed derbyall and found no error except found in regression test .

[Tomohito Nakayama]

I add one more description for the patch:
  What is done:
    3: Modify several test code to confirm 1.

[Tomohito Nakayama]

Committed .

Sending java/client/org/apache/derby/client/am/ResultSet.java
Sending java/engine/org/apache/derby/iapi/reference/SQLState.java
Sending java/engine/org/apache/derby/impl/jdbc/EmbedResultSet.java
Sending java/engine/org/apache/derby/loc/messages_en.properties
Sending java/testing/org/apache/derbyTesting/functionTests/master/DerbyNet/LOBTest.out
Sending java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/LOBTest.out
Adding java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/Stream.out
Sending java/testing/org/apache/derbyTesting/functionTests/master/LOBTest.out
Adding java/testing/org/apache/derbyTesting/functionTests/master/Stream.out
Sending java/testing/org/apache/derbyTesting/functionTests/master/connectionJdbc20.out
Sending java/testing/org/apache/derbyTesting/functionTests/suites/jdbcapi.runall
Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/LOBTest.java
Adding java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/Stream.java
Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/connectionJdbc20.java
Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/resultsetStream.java
Transmitting file data ...............
Committed revision 357435.

[Tomohito Nakayama]

There found error when using DB2 universal driver .
see http://mail-archives.apache.org/mod_mbox/db-derby-dev/200512.mbox/&lt;43A6A2BB.8040902%40basil.ocn.ne.jp>

[Tomohito Nakayama]

Description of patch :
   Add master .out file for using DB2 universal driver .

Test :
   Execute derbyall with classpath contains db2jcc.jar and db2jcc_license_c.jar , and found no error .

[Tomohito Nakayama]

I updated Component/s ...

[Tomohito Nakayama]

Committed.

Adding java/testing/org/apache/derbyTesting/functionTests/master/DerbyNet/Stream.out
Transmitting file data .
Committed revision 357966.

[Kathey Marsden]

I think this change may affect existing applicaitons. Because of the fact that the InputStream retrieved from resultset is not clean , we now throw an exception if the user gest the stream for a value in result set twice.

Typically this would mean they were getting wrong results before if they retrieved the stream more than once. Now they will get an exception which is correct but may be surprising to users that are now seeing this new symptom. A formal release note still needs to be added.

[Andrew McIntyre]

This issue has been resolved for over a year with no further movement. Closing.