Successive writes to a java.sql.Blob.setBinaryStream(long) seem to reset the file pointer

• Key: DERBY-463
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Fernanda Pizzorno
• Reporter: Laurenz Albe
• Component/s: JDBC
• Affects Version/s: 10.0.2.1
• Fix Version/s: 10.1.3.1, 10.2.1.6

Environment

Sun java full version "1.4.2_05-b04"
Linux x86
Derby is run in network server mode

Description

I have a table
PEOPLE(SEQ_ID INT NOT NULL PRIMARY KEY, PICTURE BLOB).

A row is inserted; both values are not NULL.

From inside a JDBC program, I select the Blob for update.
I then get the Blob output stream with a call to
  Blob.setBinaryStream(long)
To this stream I write several times with
  OutputStream.write(byte[], int, int)
I close the stream, update the selected row with the new Blob and commit.

The new value of the Blob now is exactly the value of the last content of the byte[],
and it is like the previous calls to write() have never taken place, or as if the file pointer
of the output stream has been reset between the calls.

A sample program follows; the size of the input file "picture.jpg" is 23237, the length
of the Blob after the program has run is 23237 % 1024 = 709

------------ sample program -------------
import java.sql.*;

class TestApp {
   private TestApp() {}

   public static void main(String[] args)
         throws ClassNotFoundException, SQLException, java.io.IOException {
      // try to load JDBC driver
      Class.forName("com.ibm.db2.jcc.DB2Driver");

      // open the input file
      java.io.InputStream instream = new java.io.FileInputStream("picture.jpg");

      // login to database
      Connection conn = DriverManager.getConnection(
            "jdbc:derby:net://dbtuxe/testdb", "laurenz", "apassword");
      conn.setAutoCommit(false);

      // select Blob for update
      PreparedStatement stmt = conn.prepareStatement(
            "SELECT PICTURE FROM PEOPLE WHERE SEQ_ID=? FOR UPDATE OF PICTURE");
      stmt.setInt(1, 1);
      ResultSet rs = stmt.executeQuery();

      // get Blob output stream
      rs.next();
      Blob blob = rs.getBlob(1);
      java.io.OutputStream outstream = blob.setBinaryStream(1l);

      // copy the input file to the Blob in chunks of 1K
      byte[] buf = new byte[1024];
      int count;
      while (-1 != (count = instream.read(buf))) {
         outstream.write(buf, 0, count);
         System.out.println("Written " + count + " bytes to Blob");
      }

      // close streams
      instream.close();
      outstream.close();

      // update Blob with new value
      String cursor = rs.getCursorName();
      PreparedStatement stmt2 = conn.prepareStatement(
            "UPDATE PEOPLE SET PICTURE=? WHERE CURRENT OF " + cursor);
      stmt2.setBlob(1, blob);
      stmt2.executeUpdate();

      // clean up
      stmt2.close();
      stmt.close();
      conn.commit();
      conn.close();
   }
}

Comments

[Fernanda Pizzorno]

Change method write (byte[], int, int) in java/client/org/apache/derby/client/am/BlobOutputStream.java. offset_ was not being incremented.

[Laurenz Albe]

While you are at it:

In my program there is a line of code:
  java.io.OutputStream outstream = blob.setBinaryStream(1l);

My original attempt was with blob.setBinaryStream(0l), as it should be in my opinion,
but then the first byte written to the Blob does not get written, i.e. the resulting Blob is one byte
short. By trial and error I found that it works when I use 1 instead of 0.

Is that also a bug or is that intended?

[Fernanda Pizzorno]

It also surprised me that it would start form 1, but it is intended to be so that the first byte is 1 and not 0.

[Oyvind Bakksjo]

The bugfix looks ok.

My comments do not concern the fix, but the patch itself:
- BlobOutputStream.java: The System.arraycopy() line's diff is just a different ordering of subtraction and addition - same behaviour as before.
- blobclob4BLOB.java: A blank line removed, and an added javadoc header without a function - should probably go into blobSetBinaryStream.java?
- blobSetBinaryStream.java:
  * No newline at end of file
  * The 'isDerbyNet' variable seems to be unused
  * Do you need the 'startJBMS()' and 'setAutoCommit()' calls after the call to testBlob1() in main()?
...and maybe this is a little picky, but:
  * Some lines are not properly indented in main().
  * Some typos in the string literals: "setBinaryStram", "mutiple" (don't forget to update the master file also).

You will usually catch things such ac unnecessary diffs yourself by looking closely at the patch before submitting it. :)

[Oyvind Bakksjo]

Committed revision 233487.

[Satheesh Bandaram]

I wonder if similar problem exists if write(int) method is used, instead of write(b[], int, int) method. It seems the offset is not incremented in this case either. Looking at ClobOutputStream.java, does this need patching too?

[Fernanda Pizzorno]

Check if clobs have a similar problem, and extend the test to use both write(byte[], int, int) and write(int).

[Fernanda Pizzorno]

- The test blobSetBinaryStreams.java was only testing the write(byte[], int, int) method for blobs. I have extendedit so that it would use both write(byte[], int, int) and write(int) for both blobs and clobs. Since it was not a blob specific test, I have renamed it to lobStreams.java.
- Similar problems as the one fixed in the previous patch (the offset was not being incremented) existed in write(int) (for clobs and blobs) and write(byte[], int, int) (for clobs). These problems have now been fixed.
- I have run derbyall successfully, except for store/encryptionKey.sql that failed, but I don't think it was related to my patch.

[Fernanda Pizzorno]

On the 08/Sep/05 I attached a new patch for this issue containing a fix for Clobs (write(byte[], int, int) and write(int)) and blobs (write(int)). I forgot to ask someone to review the patch. So could someone please review it?

[Satheesh Bandaram]

Submitted this patch. Thanks for addressing my review comments and fixing other API calls too. I do like your new test!

Looks like you haven't signed an ICLA with Apache... I would advise considering this step as your patches are getting bigger ...!

Sending java\client\org\apache\derby\client\am\BlobOutputStream.java
Sending java\client\org\apache\derby\client\am\ClobOutputStream.java
Deleting java\testing\org\apache\derbyTesting\functionTests\master\blobSetBinaryStream.out
Adding java\testing\org\apache\derbyTesting\functionTests\master\lobStreams.out
Sending java\testing\org\apache\derbyTesting\functionTests\suites\derbynetclientmats.runall
Deleting java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\blobSetBinaryStream.java
Deleting java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\blobSetBinaryStream_app.properties
Sending java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\build.xml
Sending java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\copyfiles.ant
Adding java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\lobStreams.java
Adding java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\lobStreams_app.properties
Transmitting file data ........
Committed revision 330687.

[Sunitha Kambhampati]

I have one comment on this fix.

In ClobOutputStream.java
     public void write(int b) throws java.io.IOException {
+ byte[] newByte = new byte[1];
+ newByte[0] = (byte)b;

I think it is incorrect to cast int to a byte. This will lead to incorrect results for unicode characters whose value is greater than a byte.

[Daniel John Debrunner]

Sunitha, I don't think that is a problem because the api for OutputStream.write(int b) is that it is a write of a byte, the top 24 bits are discarded.

There is a different serious problem with the class, nothing to do with this patch, it's an existing problem.
The code use String(byte[]) to create a string from a byte array. This uses the default platform encoding, which is typically not required
behaviour. Since this class is used for Clob.setAsciiStream I assume the encoding should be ascii based.
Though the performance of this class looks terrible if write(int) is used, at least five objects created for eveny byte written!

In fact this class and the client Clob have code like this everywhere, as the Clob is updated:

        clob_.string_ = clob_.string_.concat(new String(newByte));
        clob_.asciiStream_ = new java.io.StringBufferInputStream(clob_.string_);
        clob_.unicodeStream_ = new java.io.StringBufferInputStream(clob_.string_);
        clob_.characterStream_ = new java.io.StringReader(clob_.string_);

Would probably be better to create most of those objects on demand, rather than on every modification. I mean if the CLOB is modified
but the application never retrieives the ascii or character streams, what was the benefit of creating them?

[Satheesh Bandaram]

Looks like this patch has been committed for sometime... Fernanda, can you resolve the defect and after confirming the fix, can you also close the bug?

[Deepa Remesh]

Attaching 'derby-463-v10.1.diff' which ports this fix to 10.1 branch. This port is required to merge and test DERBY-683. A direct merge was not possible. Hence created a patch for v10.1.

With this patch, I ran derbyall in v10.1 with Sun jdk 1.4.2 on Windows XP. Please take a look at this patch. Thanks.

[Deepa Remesh]

Add fix version - 10.1.3

[Andrew McIntyre]

Committed to 10.1 with revision 407391.