When loaded by jvm1.6 - EmbeddedConnectionPoolDataSource is not returning a JDBC 4 compliant PooledConnection object

• Key: DERBY-2488
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: A B
• Reporter: Stan Bradbury
• Component/s: JDBC
• Affects Version/s: 10.2.2.0
• Fix Version/s: 10.3.1.4

Description

Updated the problem description to reflect the actual issue [Stan 4/5/07]: EmbeddedConnectionPoolDataSource is not returning a JDBC 4 compliant PooledConnection object

When loaded using JVM 1.6 the JDBC 3.0 data source (org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource) is reporting a JDBC spec compliance level of 4. The DatabaseMetaData.getJDBCMajorVersion method should return the
correct spec compliance level (3) for the JDBC 3.0 data source. The application server that relies on the reported spec compliance level to determine if a statement event listener should be registered will encounter a failure when attemptting to register an event listener with the JDBC 3.0 driver.

Note that the JDBC 4.0 metadata call ..getMetaData().supportsStoredFunctionsUsingCallSyntax() is executed by the JDBC 3.0 driver. This doesn't bother me but seems curious that this is supported by a JDBC 3.0 driver.

Attached is a test case based on the test code from DERBY-8862 (JDBC 4 driver reports incorrect spec complieance level).

Running the test:
set JAVA_HOME to a 1.6 JVM
Compile program
run command: ${JAVA_HOME}/java showJDBCdsMetaDataLis <path-to-database> 3.0

Using JAVA 6.0

> "%JAVA_HOME%\bin\java" showJDBCdsMetaDataLis "c:\tstdb102" 3.0
 . . JDBC 3 or less: Using jdbc.EmbeddedConnectionPoolDataSource
Database product: Apache Derby
Database version: 10.2.2.0 - (485682)
Driver name: Apache Derby Embedded JDBC Driver
Driver version: 10.2.2.0 - (485682)
JDBC driver major version: 4
JDBC driver minor version: 0
JDBC 4.0 metaData - SFusingCall: true
 >> exception thrown:
A non SQL error occured.
java.lang.AbstractMethodError: org.apache.derby.jdbc.EmbedPooledConnection.addStatementEventListener(Ljavax/sql/Statemen
tEventListener;)V
        at showJDBCdsMetaDataLis.main(showJDBCdsMetaDataLis.java:103)

Comments

[Stan Bradbury]

load the JDBC 4.0 datasource driver when args[1] = 6.0
If any other value is passed the JDBC 3.0 datasource driver is used

[Stan Bradbury]

Attaching again: Forgot to Grant license to ASF. Thought it might be important

[Daniel John Debrunner]

I think the bug here is that EmbeddedConnectionPoolDataSource is not returning a JDBC 4 compliant PooledConnection object in JDK 6.

These "older" data source implementations do return JDBC 4 objects (at least they are meant to), it's just that they themselves cannot implement the JDBC apis such as Wrapper and CommonDataSource because these new interfaces do not exist on older jvms. They are not just JDBC 3 versions of the driver.

[Stan Bradbury]

Thanks, Dan, for the information. I realized the results of this test could be interpreted two ways and it looks like I recorded the wrong interpretation. Unless you or someone corrects me (I am still confused on the subject) I will change the description of this issue to be:

When loaded by jvm1.6, the jdbc.EmbeddedConnectionPoolDataSource does not support registering a JDBC 4.0 Statement Event Listener.

The documentation, however, seems to say there are two versions, so I will also file documentation sub-tasks related to this once I am sure I understand this correctly. Below are a few of the statements that lead me to believe there was a version 3.0 and 4.0 datasource.
...If I now understand correctly people should always reference EmbeddedConnectionPoolDataSource and there is no need to reference EmbeddedConnectionPoolDataSource40 (except as a workaround to a bug like this one) .
 >> Is this correct?

Documentation talking about different implementations/variants of the datasources:
JAVADOC
 EmbeddedConnectionPoolDataSource40 supports the JDBC 4.0 specification for the J2SE 6.0 Java Virtual Machine environment. Use EmbeddedConnectionPoolDataSource if your application runs in the following environments:

    * JDBC 3.0 - Java 2 - JDK 1.4, J2SE 5.0
    * JDBC 2.0 - Java 2 - JDK 1.2,1.3

Developers Guide:
Each of these DataSources has two variants. Use the first variant if your application run
on JDK 1.5 or lower. Use the second variant (the one whose class name ends with "40")
if your application runs on JDK 1.6 or higher.
• org.apache.derby.jdbc.EmbeddedDataSource and
org.apache.derby.jdbc.EmbeddedDataSource40 ....

[Daniel John Debrunner]

No, there are two data sources versions, the 40 ones and the older ones.

In a JDK 1.4/1.5 environment the applications must use the old ones (e.g. EmbeddedConnectionPoolDataSource)

In JDK 6 applications should be using EmbeddedConnectionPoolDataSource40.

The situation you are seeing is that the old version is being run in JDK 6, which of course could happen for existing applications.
In this case the api presented is a bit of a mix-up, it is not coded to be a JDBC 3.0 data source.
It will (should) return JDBC 4 objects even though the object it self does not satisify the JDBC 4.0 interfaces. Ie. EmbeddedConnectionPoolDataSource does not implement Wrapper and CommonDataSource.

I think the documentation is correct, but silent on what happens when using the old versions in JDK 6.

[A B]

I read Dan's comments on this issue and I (perhaps incorrectly) interpret them to mean that the fix for this problem is to simply ensure that EmbeddedConnectionPoolDataSource returns JDBC 4 objects when the app is running with JDK 6. That said, it looks like there is only one JDBC 4 object that can be returned from EmbeddedConnectionPoolDataSource, and that object is a PooledConnection. So I made the attached change (d2488_notTested_v1.patch), which is pretty much just a one-liner to return a JDBC 4 PooledConnection when running with JDK 6.

The patch does a similar thing in EmbeddedXADataSource just because that *seems* like a similar situation. I don't know for sure if that's necessary, but it looks like it could be.

As the name implies I have not fully tested this patch. All I've done is run the repro attached to this issue and verified that the _v1 patch causes the repro to run successfully, even with an input argument that is not "6.0":

  > java showJDBCdsMetaDataLis dbdb 2.4
   . . JDBC 3 or less: Using jdbc.EmbeddedConnectionPoolDataSource
  Database product: Apache Derby
  Database version: 10.3.0.0 alpha - (1)
  Driver name: Apache Derby Embedded JDBC Driver
  Driver version: 10.3.0.0 alpha - (1)
  JDBC driver major version: 4
  JDBC driver minor version: 0
  JDBC 4.0 metaData - SFusingCall: true
  Event: statementClosed a816c00e-0111-b830-0576-0000003d4c38

Please note, though, that I am not at all familiar with this area of code, so it's possible I'm being far too naive here. If that's the case then I'm hoping someone can elaborate a bit more on what the correct approach might be...

[A B]

Okay, so d2488_noTest_v1 is not a valid patch. It was fine when I applied and built, but when I clobbered and then built there are failures (because the 40 classes can't be built with jdk14). Oops, silly me; sorry for th noise.

Still curious to know, though, if this is the correct approach *functionally* speaking? I.e. we need to return JDBC 4 objects from the non-40 data sources?

[A B]

Attaching a second patch that has the same functional behavior as the first one, except that it uses reflection to create the JDBC 4 object so that the non-JDBC 4 classes will still compile with 1.4.

I did a full ant clobber and ant all with this patch and the build finishes cleanly. I also ran suites.All and derbyall on Red Hat Linux with ibm142 with no issues. I haven't run those suites with jdk16 yet because I don't have a machine available with that jvm at the moment.

Before going any further with this I'm hoping someone can offer input as to whether or not this is the correct approach. Or to be more explicit, Dan, do these changes correlate with your comments above, or am I totally off-track here?

[Daniel John Debrunner]

Your approach is what I was describing though I think there is a somewhat cleaner way that doesn't require reflection.

InternalDriver is the class that is the factory for all JDBC object, e.g. Connections, ResultSets etc. It has methods that return the correct level of JDBC object for a given JVM environment. This is because the specific version of InternalDriver is active in the JVM, e.g. Driver30 for JDBC 3.0 and Driver40 for JDBC 4.0.

So the first thought would be to add a getNewPooledConnection(...) and getNewXAConnection(...) to InternalDriver, and then have the various implementations return the correct type for their JDBC level, thus no reflection.

However, InternalDriver needs to work on JSR169 as it's the factory for objects there as well, and JSR169 does not support PooledConnection or XAConnection,
so the methods cannot be added there.

What could be done is to add the newPooledConnection(...) and newXAConnection(...) methods to Driver30 and then have the data source use those methods to get the objects, e.g.


   ((Driver30) findDriver()).getNewPooledConnection(...);

That code would then work on JDBC 3.0 and JDBC 4.0 with no need for the 40 versions of the DataSource to have specific methods to get new objects, instead that code would be in Driver40.

   

[A B]

Thank you for the feedback, Dan. The approach you've mentioned does indeed sound cleaner. I will work with this some more to see if I can implement your suggestions.

[Stan Bradbury]

Updated the description to accurately reflect the problem as it is now understood.

[A B]

Attaching d2488_v3.patch, which is an attempt to implement Dan's suggested approach. This new approach is far cleaner than the original--so thank you to Dan for the great feedback.

While working on this I discovered that the ClientXADataSource class suffers from the same problem as the Embedded classes, so I made similar changes there. Note that the ClientConnectionPoolDataSource file was already doing something very similar to what Dan had suggested for Embedded, except that instead of using "Driver30" it uses "ClientDriver.getFactory()". So that particular data source was working correctly even before my changes. Given that, I have now updated the ClientXADataSource class to match the behavior of ClientConnectionPoolDataSource.

This patch also includes a new test, jdbc4/JDBC4FromJDBC3DataSourceTest, which verifies that JDBC 4 connections will be returned from JDBC 3 data sources if the JDK in use is JDK 1.6. I had to make some minor modifications to several classes in the junit/ directory, but hopefully none of these is a problem. In a clean trunk the new test shows 3 failures (the ClientConnectionPoolDataSource test passes, as mentioned above); after applying d2488_v3.patch, the test runs cleanly.

I ran derbyall and suites.All on a SUSE Linux 10.0 machine and there were the following failures, which also appear in the Tinderbox:

  - derbyall: lang/grantRevokeDDL.sql
  - suites.All: GroupByExpressionTest.testMiscExpressions
  - suites.All: SecurityPolicyReloadingTest
  - suites.All: UpdateCursorTest.testVirtualMemoryHeap

There was also a failure in derbynet/testconnection.java, which does not show up in the tinderbox:

  9d8
  < Unable to find host: ihave-inmyname.com.
  Test Failed.

When I ran this on my Windows 2000 laptop the diff was slightly different:

  9 del
  < Unable to find host: ihave-inmyname.com.
  9a9
  > Could not connect to Derby Network Server on host ihave-inmyname.com, port 1527.

I did an "ant clobber" and rebuilt just to make sure, but the test still fails. I then tried running it on a clean, fully-synced client and it still failed (with the same diff). So I don't think this is related to my changes...

Review comments and any other feedback would be much appreciated, especially since (as I mentioned earlier) this area of code is very new to me...

[A B]

Attaching a _v4 patch that is identical to _v3 in terms of engine/client changes. The only difference is that I have made the following tweaks to the new test:

  1. Wired it into jdbc4/_Suite.java (forgot to do that with _v3)
  2. Created a new convenience method on TestConfiguration for forcing use of JDBC 3 datasources (instead of putting that code inside the new test).
  3. Updated the test (jdbc4/JDBC4FromJDBC3DataSourceTest) to use the new convenience method.

Note that the convenience method on TestConfiguration wraps the test inside a CleanDatabaseTestSetup configuration. This ensures that the target database ("wombat") will always exist before the fixtures are run (which was not the case for the _v3 test).

[A B]

I haven't heard any comments (good nor bad) about the _v4 patch for this issue. I hope to re-run the regression suites later today to make sure everything is still okay (esp. since some DataSource tests were added to suites.All aftet I posted the patch), and if everything passes then I will likely commit d2488_v4.patch before the end of today (Friday).

If people have comments/feedback after that, I can of course address them in subsequent patches.

[A B]

derbyall and suites.All both ran cleanly for me after a full sync / clobber / rebuild. So I removed an extraneous import from the _v4 patch and committed with svn # 531129:

  URL: http://svn.apache.org/viewvc?view=rev&rev=531129

Marking the issue as resolved. Thanks again to Dan for his input on how to best fix this.

[Stan Bradbury]

Catching up on Closing my reported issues. Thanks to Dyre for the workflow reminder today.