test blobclob4BLOB fails on OS 390

• Key: DERBY-575
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Minor
• Assignee: Myrna van Lunteren
• Reporter: Myrna van Lunteren
• Component/s: Test
• Affects Version/s: 10.1.1.0
• Fix Version/s: 10.1.3.1, 10.2.1.6

Description

The subtest clobNegativeTest_Derby265 in test jdbcapi/blobclob4BLOB assumes that the length of the file used (aclob.txt) is 300000 but on OS 390 (z/OS), after conversion to EBCDIC, the length is 297000. The test should be modified to not use a hardcoded size.

Comments

[Daniel John Debrunner]

Or should the test read the file with a fixed encoding so that the size remains as the test expects.

Not sure though what is doing the conversion to EBCDIC, when you run on OS 390, hopefully you don't have to covert evryrthing before running the tests.

[Myrna van Lunteren]

Unfortunately, you *do* have to convert most of the text files used by the various tests.
Certainly all master files, and for instance text files for import/export.

I unraveled derbyTesting.jar, used native2ascii -reverse in a script on the os390 machine to convert to Cp1047 and jar-ed it back up (for space).

[Daniel John Debrunner]

I would see the need to convert as a bug in the test harness. I think the harness should be reading the text files in a fixed format so that the test pass regardless of the default encoding of the platform.

[Myrna van Lunteren]

Logging a new bug for the test harness adjustment, and when that is done I'll see if the test needs further adjustments.

[Myrna van Lunteren]

After doing work following from DERBY-903 and working with the test harness improvement suggested with DERBY-683, I now better understand what was the problem with this subtest of blobclob4BLOB is.
It's the following section:
            FileReader fr = new FileReader(fileName[4]);
            ps.setInt(1, i);
            ....
            ps.setCharacterStream(6, new java.io.BufferedReader(fr),300000);
            ps.executeUpdate();
The api doc says:
" The constructors of this class assume that the default character encoding and the default byte-buffer size are appropriate. To specify these values yourself, construct an InputStreamReader on a FileInputStream. "
Thus, to make blobclob4BLOB pass with -DderbyTesting.encoding=UTF-16 (you need the improved patch for test harness from DERBY-683 to run tests like that) I think this section needs to be changed to:
            File file = new File(fileName[4]);
            InputStream fileIS = new FileInputStream(file);
            Reader fr = new InputStreamReader(fileIS, "US-ASCII");
            ps.setInt(1, i);
           ...
            ps.setCharacterStream(6, fr, 300000);

This then passes with -DderbyTesting.encoding=UTF-16 with embedded (after also fixing DERBY-902) and DerbyNet. DerbyNetClient has some problem which I think may be due to DERBY-901

I will run some more tests, then submit a patch for DERBY-903; if that proves ok & gets committed, this particular bug can be closed and unlinked from DERBY-658.

[Myrna van Lunteren]

This has now been fixed with DERBY-903, revision 379643

[Myrna van Lunteren]

reopening issue to follow up on further work needed - the test still fails on zOS with ibm15 after fix for DERBY-658.

I need to analyse it further.

Also, the test lobStreams.java fails (see DERBY-683), and it seems to me the failures may be similar.

[Myrna van Lunteren]

Attached is a patch for this issue - DERBY-575_102_20060424.*

It contains changes to and for the tests:
jdbcapi/blobclob4BLOB.java
jdbcapi/lobStreams.java
tools/ieptests.sql

The changes boil down to ensuring that appropriate mechanisms are used to accessing supported files - and in the case of blobclob4BLOB and lobStreams, I ended up copying the supportfiles because they were used by other tests with another encoding. So now, the .utf tests are copied unchanged, i.e. in UTF-8 encoding, and the .txt files are copied into local encoding.

[Myrna van Lunteren]

Patch for 10.2 is attached, a patch for 10.1 will follow.

[Andrew McIntyre]

Committed to trunk with revision 397028.

[Myrna van Lunteren]

Patch DERBY-575_101_20060502.* is the 10.1 patch for this issue. I really did an svn rename for the .txt files to .utf files, but that's not obvious. So, applying & committing this patch may need some extra care.

[Andrew McIntyre]

Thanks for making the patch for 10.1, as I was unable to svn merge it. Committed to 10.1 with revision 399021.