Issue Details (XML | Word | Printable)

Key: DERBY-500
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Sunitha Kambhampati
Reporter: Peter Kovgan
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Derby

Update/Select failure when BLOB/CLOB fields updated in several rows by PreparedStatement using setBinaryStream and setCharacterStream

Created: 09/Aug/05 05:38 PM   Updated: 09/Nov/05 06:09 AM
Return to search
Component/s: JDBC
Affects Version/s: 10.1.1.0
Fix Version/s: 10.1.2.1, 10.2.1.6

Time Tracking:
Not Specified

File Attachments:
  Size
Text File Licensed for inclusion in ASF works Derby500.diff.txt 2005-10-18 04:35 PM Sunitha Kambhampati 38 kB
Text File Licensed for inclusion in ASF works Derby500.stat.txt 2005-10-18 03:52 PM Sunitha Kambhampati 0.8 kB
Text File Licensed for inclusion in ASF works Doc_JdbcImpl_paper.diff.txt 2005-10-21 01:23 PM Sunitha Kambhampati 2 kB
Text File Licensed for inclusion in ASF works V10.1_Derby500.diff.txt 2005-10-20 07:07 AM Sunitha Kambhampati 40 kB
Text File Licensed for inclusion in ASF works V10.1_Derby500.stat.txt 2005-10-20 07:07 AM Sunitha Kambhampati 1 kB
Environment: Windows 2000, java SDK 1.4

Resolution Date: 21/Oct/05 12:59 PM


 Description  « Hide
I have table contained BLOB and CLOB fields:

Create table string is:

private static final String CREATE = "CREATE TABLE ta (" +
            "ta_id INTEGER NOT NULL," +
            "mname VARCHAR( 254 ) NOT NULL," +
            "mvalue INT NOT NULL," +
            "mdate DATE NOT NULL," +
            "bytedata BLOB NOT NULL," +
            "chardata CLOB NOT NULL," +
            "PRIMARY KEY ( ta_id ))";


Then I insert 2000 rows in the table.



Then I update all 2000 rows by command:

private static final String UPDATE = "UPDATE ta " +
     "SET bytedata=? ,chardata=? " +
     "WHERE mvalue=?";

/**create blob and clob arrays**/
        int len1 = 10000;//for blob length data
        int len2 = 15000;//for clob length data
        byte buf [] = new byte[len1];
        for(int i=0;i<len1;i++){
         buf [i] = (byte)45;
        }
        ByteArrayInputStream bais = new ByteArrayInputStream(buf);
        
        char[] bufc = new char[len2];
        for (int i = 0; i < bufc.length; i++) {
         bufc[i] = (char)'b';
}
        CharArrayReader car = new CharArrayReader(bufc);
/***/
PreparedStatement pstmt = connection.prepareStatement(UPDATE);
pstmt.setBinaryStream(1,bais, len1);
pstmt.setCharacterStream(2,car, len2);
pstmt.setInt(3,5000);
int updated = pstmt.executeUpdate();
pstmt.close();
System.out.printlen("updated ="+updated );


all 2000 rows updated , because I receive output : updated =2000

But If I run select (SELECT bytedata ,chardata FROM ta) after update, select failed with error:

ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted to
 read more data than was originally stored
        at org.apache.derby.iapi.error.StandardException.newException(StandardEx
ception.java)
        at org.apache.derby.impl.store.raw.data.StoredPage.readRecordFromArray(S
toredPage.java)
        at org.apache.derby.impl.store.raw.data.StoredPage.restoreRecordFromSlot
(StoredPage.java)
        at org.apache.derby.impl.store.raw.data.BasePage.fetchFromSlot(BasePage.
java)
        at org.apache.derby.impl.store.access.conglomerate.GenericScanController
.fetchRows(GenericScanController.java)
        at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapS
can.java)
        at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(
BulkTableScanResultSet.java)
        at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCo
re(BulkTableScanResultSet.java)
        at org.apache.derby.impl.sql.execute.NestedLoopJoinResultSet.getNextRowC
ore(NestedLoopJoinResultSet.java)
        at org.apache.derby.impl.sql.execute.NestedLoopLeftOuterJoinResultSet.ge
tNextRowCore(NestedLoopLeftOuterJoinResultSet.java)
        at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRow
Core(ProjectRestrictResultSet.java)
        at org.apache.derby.impl.sql.execute.SortResultSet.getRowFromResultSet(S
ortResultSet.java)
        at org.apache.derby.impl.sql.execute.SortResultSet.getNextRowFromRS(Sort
ResultSet.java)
        at org.apache.derby.impl.sql.execute.SortResultSet.loadSorter(SortResult
Set.java)
        at org.apache.derby.impl.sql.execute.SortResultSet.openCore(SortResultSe
t.java)
        at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(BasicN
oPutResultSetImpl.java)
        at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPre
paredStatement.java)
        at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedState
ment.java)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Em
bedPreparedStatement.java)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(EmbedPrepar
edStatement.java)
        at com.beep_beep.dbtest.complex.Benchmark.testSelect(Unknown Source)
        at com.beep_beep.dbtest.complex.Benchmark.executeSimplestBigTable(Unknown Sour
ce)
        at com.beep_beep.dbtest.complex.Benchmark.testBigTable(Unknown Source)
        at com.beep_beep.dbtest.complex.Benchmark.executeDegradationBenchmark(Unknown
Source)
        at com.beep_beep.dbtest.complex.Benchmark.main(Unknown Source)


From the stack trace and from console I see that Update passed, but error was raised in Select after Update.


When I try the same update, but with difference(I changed WHERE clause, causing update only 1 row):
private static final String UPDATE = "UPDATE ta " +
     "SET bytedata=? ,chardata=? " +
     "WHERE mname=?";

PreparedStatement pstmt = connection.prepareStatement(UPDATE);
pstmt.setBinaryStream(1,bais, len1);
pstmt.setCharacterStream(2,car, len2);
pstmt.setInt(3,"PETER");
int updated = pstmt.executeUpdate();
pstmt.close();
System.out.printlen("updated ="+updated );

Only 1 row updated , because I receive output : updated =1

In this case I have NO errors in select(the same as previous) .

My assumption:
It seems that Update receives ByteArrayInputStream and updates correctly only 1 row, then all rows updated by some
incorrect value(may be because ByteArrayInputStream reached its end in first update), causing select failure.

I tested PointBase by the same test and PointBase passed this stage without errors, no matter how many rows was updated.
So I think it is a bug.

Thank you.
















 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Repository Revision Date User Message
ASF #326307 Wed Oct 19 00:11:30 UTC 2005 mikem committing fix for DERBY-500 on behalf of: Sunitha Kambhampati

 Background :
 In Derby, when a stream is set as a parameter value, the wrapper stream object used for character data is ReaderToUTF8Stream
 and for binary data it is RawToBinaryFormatStream.Both these stream objects on read() return data in a format that is used to store the respective datatype value. E.g in case of char, the characters read from the user stream are converted using UTF-8 derby specific encoding and read calls return
 the data as expected by store layer. Beginning 2 bytes either have the utflen or has zeroes, or if it is a long string, then the value is ended with the special marker 0xE0 , 0x00, 0x00. For binary data, the stream data is prepended with 4 zeroes.

 Problem:
 once,the stream has been read fully and end of file reached, further read() returns a -1. If a stream is re-read, it returns a -1 which is incorrect data. E.g.in the repro for DERBY-500, the update statement has multiple rows that qualify and since the stream parameter is used; the first row gets updated with the correct value and the stream is drained. For the subsequent rows, the read from the stream parameter value returns -1 and thus is updated with incorrect data.When retrieving the row back, the format of the fields is incorrect and thus the exception.
 __________
 This patch

 1. adds changes to RawToBinaryFormatStream and ReaderToUTF8Stream to throw an EOFException if stream is re-read.
 If a stream value has been fully read and end of file reached, any further reads on the stream object will result in an EOFException. This seems reasonable and more correct than using incorrect values.
 Adds a new error message - 'Stream has already been read and end-of-file reached and cannot be re-used.'

 2. changes to RememberBytesInputStream to keep track of the stream state and not call read on the stream objects once eof is reached.

 3. Fix a bug in StoredPage.logColumn related to streams. In one particular scenario, column was not being set to RememberBytesInputStream object and thus losing the data that would be read from stream into RememberBytesInputStream.

 4. adds testcases to store/streamingColumn.java and lang/forbitdata.java


 Also note
 - This fix affects cases when a stream is re-used in which case an exception will be thrown.
 So code that reads the stream once and materializes it will not be affected. E.g. Currently in case of char,varchar,long varchar, streams are materialized and this will work fine as before.
Files Changed
MODIFY /db/derby/code/trunk/java/engine/org/apache/derby/impl/jdbc/RawToBinaryFormatStream.java
MODIFY /db/derby/code/trunk/java/engine/org/apache/derby/impl/jdbc/ReaderToUTF8Stream.java
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/streamingColumn.java
MODIFY /db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/StoredPage.java
MODIFY /db/derby/code/trunk/java/engine/org/apache/derby/iapi/reference/SQLState.java
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/streamingColumn.out
MODIFY /db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/RememberBytesInputStream.java
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/forbitdata.out
MODIFY /db/derby/code/trunk/java/engine/org/apache/derby/loc/messages_en.properties
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/forbitdata.java

Repository Revision Date User Message
ASF #327012 Thu Oct 20 22:18:36 UTC 2005 kmarsden DERBY-500 - Update/Select failure when BLOB/CLOB fields updated in several rows by PreparedStatement using setBinaryStream and setCharacterStream

Sunitha merged r326307 from trunk manually because of conflicts.

Contributed by Sunitha Kambhampati (ksunithaghm@gmail.com)
Files Changed
MODIFY /db/derby/code/branches/10.1/java/engine/org/apache/derby/impl/jdbc/RawToBinaryFormatStream.java
MODIFY /db/derby/code/branches/10.1/java/testing/org/apache/derbyTesting/functionTests/tests/store/streamingColumn.java
MODIFY /db/derby/code/branches/10.1/java/engine/org/apache/derby/iapi/reference/SQLState.java
MODIFY /db/derby/code/branches/10.1/java/engine/org/apache/derby/impl/jdbc/ReaderToUTF8Stream.java
MODIFY /db/derby/code/branches/10.1/java/testing/org/apache/derbyTesting/functionTests/master/streamingColumn.out
MODIFY /db/derby/code/branches/10.1/java/engine/org/apache/derby/impl/store/raw/data/StoredPage.java
MODIFY /db/derby/code/branches/10.1/java/testing/org/apache/derbyTesting/functionTests/tests/store/streamingColumn_derby.properties
MODIFY /db/derby/code/branches/10.1/java/engine/org/apache/derby/loc/messages_en.properties
MODIFY /db/derby/code/branches/10.1/java/testing/org/apache/derbyTesting/functionTests/tests/lang/forbitdata.java
MODIFY /db/derby/code/branches/10.1/java/engine/org/apache/derby/impl/store/raw/data/RememberBytesInputStream.java
MODIFY /db/derby/code/branches/10.1/java/testing/org/apache/derbyTesting/functionTests/master/forbitdata.out

Repository Revision Date User Message
ASF #331887 Tue Nov 08 20:55:52 UTC 2005 jta DERBY-500 Applied Sunitha Kambhampati's Derby web site patch for
          http://db.apache.org/derby/papers/JDBCImplementation.html .
Files Changed
MODIFY /db/derby/site/trunk/src/documentation/content/xdocs/papers/JDBCImplementation.html
MODIFY /db/derby/site/trunk/build/site/papers/JDBCImplementation.html