Derby
  1. Derby
  2. DERBY-4491

The network client changes UDTs into Strings and returns their type as LONGVARBINARY.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.0.2.0, 10.0.2.1, 10.1.1.0, 10.1.2.1, 10.1.3.1, 10.2.1.6, 10.2.2.0, 10.3.1.4, 10.3.2.1, 10.3.3.0, 10.4.1.3, 10.4.2.0, 10.5.1.1, 10.5.2.0, 10.5.3.0
    • Fix Version/s: 10.6.1.0
    • Component/s: JDBC
    • Issue & fix info:
      Patch Available
    • Bug behavior facts:
      Embedded/Client difference

      Description

      This is a pre-existing bug which seems to have been with Derby since the beginning. Some of the columns in the system tables (e.g., SYS.SYSALIASES.ALIASINFO) contain objects. If you select these columns:

      1) In the embedded client you will get the correct results. You will get the objects in these columns. In addition, the ResultSetMetaData for these columns will correctly report that the columns have type JAVA_OBJECT and will give a reasonable type name (the class name for the object in the column).

      2) However, in the network client, you will get the wrong results. ResultSet.getObject() will return Strings rather than the original objects. In addition, the ResultSetMetaData for these columns will incorrectly report that their type is LONGVARBINARY.

      1. derby-4491-02-aa-supportsUDTs.diff
        0.5 kB
        Kristian Waagan
      2. derby-4491-01-ad-networkTransport.diff
        96 kB
        Rick Hillegas
      3. derby-4491-01-ab-networkTransport.diff
        74 kB
        Rick Hillegas

        Issue Links

          Activity

          Hide
          Rick Hillegas added a comment -

          Here are some thoughts about how to address this issue.

          DRDA does support user defined types and our existing protocol flows maintain a placeholder for UDT information. Right now we plug a null into that placeholder. For more detail on this support, see the February 2007 version of the DRDA spec, Volume 1 (Data Definition and Exchange), section 5.6.4.10 (SQL Descriptor User-Defined Type Group Description, aka SQLUDTGRP).

          The DRDA support for user defined types, however, is not rich enough to describe the Java user defined types defined by part 13 of the ANSI/ISO SQL Standard. These are the user defined types identified by the JDBC type code java.sql.Types.JAVA_OBJECT. DRDA support only covers SQL distinct, struct, and ref types, that is, those which map to the STRUCT, DISTINCT, and REF constants in java.sql.Types. There is no corresponding DRDA constant mapping to java.sql.Types.JAVA_OBJECT. In addition, the DRDA protocol does not convey the Java class name needed to fulfill the JDBC contract for ResultSetMetaData.getColumnClassName() and ParameterMetaData.getParameterClassName().

          Therefore, in order to fulfill the JDBC contract, we will have to extend the DRDA protocol. I propose the following:

          1) For compatibility reasons, we will maintain the old, incorrect behavior if either the client or the server is NOT Derby code at level 10.6 or higher.

          2) However, if the client and server are both Derby code at version 10.6 or higher, then the user will see the embedded behavior. Internally, we will implement this with Derby-specific extensions to DRDA. This affects the following methods:

          ResultSet.getObject() - Will return the UDT object rather than the result of calling toString() on it.

          PreparedStatement.setObject() - Will accept UDTs if the parameter is typed as JAVA_OBJECT. The object being set must be an instance of the Java class which was bound to the UDT by the CREATE TYPE statement.

          ResultSetMetaData.getColumnType() and ParameterMetaData.getParameterType() - Will return JAVA_OBJECT rather than LONGVARBINARY.

          ResultSetMetaData.getColumnTypeName() and ParameterMetaData.getParameterTypeName() - Will return the fully qualified name of the UDT rather than LONG VARCHAR FOR BIT DATA. However, for our legacy user defined types (the ones stored in the system tables), we will continue to follow the embedded practice of returning the class name without any schema qualifier. So, for a column of type Price, we will return what is required by the JDBC spec

          "APP"."PRICE"

          but for SELECT ALIASINFO FROM SYS.SYSALIASES, we will return

          org.apache.derby.catalog.AliasInfo

          This behavior for the system columns seems to fall short of the JDBC contract. However, these are special types and I am content to leave them alone. If we are not happy about this approach for the system columns, then we should open a new JIRA and come up with a better common behavior for both embedded and network usage.

          ResultSetMetaData.getColumnClassName() and ParameterMetaData.getParameterClassName() - Will return the name of the class bound to the UDT when it was defined.

          ResultSetMetaData.getColumnPrecision() and ParameterMetaData.getPrecision() - Will return 0 as in the embedded case.

          ResultSetMetaData.getColumnScale() and ParameterMetaData.getScale()- Will return 0 as in the embedded case.

          ResultSetMetaData.getColumnDisplaySize() - Will return 15 as in the embedded case. This is Derby's default column display size and seems arbitrary to me. However, I find it hard to argue for some other number. If we decide that some other number is better, then we should open a new JIRA and use the same number for both embedded and network situations.

          Show
          Rick Hillegas added a comment - Here are some thoughts about how to address this issue. DRDA does support user defined types and our existing protocol flows maintain a placeholder for UDT information. Right now we plug a null into that placeholder. For more detail on this support, see the February 2007 version of the DRDA spec, Volume 1 (Data Definition and Exchange), section 5.6.4.10 (SQL Descriptor User-Defined Type Group Description, aka SQLUDTGRP). The DRDA support for user defined types, however, is not rich enough to describe the Java user defined types defined by part 13 of the ANSI/ISO SQL Standard. These are the user defined types identified by the JDBC type code java.sql.Types.JAVA_OBJECT. DRDA support only covers SQL distinct, struct, and ref types, that is, those which map to the STRUCT, DISTINCT, and REF constants in java.sql.Types. There is no corresponding DRDA constant mapping to java.sql.Types.JAVA_OBJECT. In addition, the DRDA protocol does not convey the Java class name needed to fulfill the JDBC contract for ResultSetMetaData.getColumnClassName() and ParameterMetaData.getParameterClassName(). Therefore, in order to fulfill the JDBC contract, we will have to extend the DRDA protocol. I propose the following: 1) For compatibility reasons, we will maintain the old, incorrect behavior if either the client or the server is NOT Derby code at level 10.6 or higher. 2) However, if the client and server are both Derby code at version 10.6 or higher, then the user will see the embedded behavior. Internally, we will implement this with Derby-specific extensions to DRDA. This affects the following methods: ResultSet.getObject() - Will return the UDT object rather than the result of calling toString() on it. PreparedStatement.setObject() - Will accept UDTs if the parameter is typed as JAVA_OBJECT. The object being set must be an instance of the Java class which was bound to the UDT by the CREATE TYPE statement. ResultSetMetaData.getColumnType() and ParameterMetaData.getParameterType() - Will return JAVA_OBJECT rather than LONGVARBINARY. ResultSetMetaData.getColumnTypeName() and ParameterMetaData.getParameterTypeName() - Will return the fully qualified name of the UDT rather than LONG VARCHAR FOR BIT DATA. However, for our legacy user defined types (the ones stored in the system tables), we will continue to follow the embedded practice of returning the class name without any schema qualifier. So, for a column of type Price, we will return what is required by the JDBC spec "APP"."PRICE" but for SELECT ALIASINFO FROM SYS.SYSALIASES, we will return org.apache.derby.catalog.AliasInfo This behavior for the system columns seems to fall short of the JDBC contract. However, these are special types and I am content to leave them alone. If we are not happy about this approach for the system columns, then we should open a new JIRA and come up with a better common behavior for both embedded and network usage. ResultSetMetaData.getColumnClassName() and ParameterMetaData.getParameterClassName() - Will return the name of the class bound to the UDT when it was defined. ResultSetMetaData.getColumnPrecision() and ParameterMetaData.getPrecision() - Will return 0 as in the embedded case. ResultSetMetaData.getColumnScale() and ParameterMetaData.getScale()- Will return 0 as in the embedded case. ResultSetMetaData.getColumnDisplaySize() - Will return 15 as in the embedded case. This is Derby's default column display size and seems arbitrary to me. However, I find it hard to argue for some other number. If we decide that some other number is better, then we should open a new JIRA and use the same number for both embedded and network situations.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-4491-01-ab-networkTransport.diff. This is a first rev of a patch to address this bug. This patch is not ready to commit yet: I need to write new regression tests to track the new behavior. However, the existing regression tests pass cleanly for me.

          This patch does the following:

          1) Makes network JDBC behavior mimic the embedded JDBC behavior as described in the previous comment. The behaviors agree when the client and server are Derby code at level 10.6 or higher. Otherwise, the network behavior continues to be what it was in previous releases--that is, the buggy behavior described by this JIRA.

          2) To satisfy the JDBC contract, this patch implements a Derby-only extension to DRDA at the point in the metadata exchange where we currently write a null SQLUDTGRP. The extended protocol is described in the header comment for DRDAConnThread.writeSQLUDTGRP().

          3) This patch adds DRDA and DB2 type codes for UDTs. The DRDA type codes are defined in the SQLUDTGRP section of the DRDA spec (Volume 1, section 5.6.4.10 SQL Descriptor User-Defined Type Group Description). Note that these DRDA type codes for UDTs do not appear in the summary 5-11 table in Volume 1--I believe that is an oversight on the part of the editors. I could not find DB2 type codes for UDTs. If someone knows what these are, that would be an improvement to this patch. In the meantime, I have invented type codes in a part of the code space which I think will not collide with DB2 evolution during our lifetimes. Since this is a Derby-only extension, I believe we are safe here regardless of how DB2 evolves.

          4) Only small UDTs can be transported across DRDA right now. A small UDT cannot serialize to more than 32767 (0x7FFF) bytes, the maximum size of a DRDA network protocol buffer. In the future, we may want to add support for big UDTs. My feeling is that we will want to give the type designer some way to declare that a UDT is big--for instance, the UDT's Java class could implement a vacuous org.apache.derby.types.BigUDT interface. Based on that declaration, we could stream the big UDTs across the network. Based on this BigUDT declaration, we could also optimize Store and language support for these values and we could build support for SQL UDT locators. All of that work, however, requires community discussion and falls outside the scope of this JIRA.

          5) In order to support UDT serialization across the network, two classes have been moved into the common part of the code tree so that the network client can use them: DynamicByteArrayOutputStream and InputStreamUtil.

          More work is needed:

          A) Regression and compatibility tests need to be added.

          B) DERBY-4449 needs to be fixed so that we can use UDTs as output parameters in db procs.

          Touches the following files:

          M java/engine/org/apache/derby/loc/messages.xml
          M java/shared/org/apache/derby/shared/common/reference/SQLState.java

          New error messages.

          M java/shared/org/apache/derby/shared/common/reference/JDBC30Translation.java
          M java/engine/org/apache/derby/iapi/types/DataTypeUtilities.java
          M java/engine/org/apache/derby/catalog/types/TypeDescriptorImpl.java

          Replaced some magic numbers with manifest constants.

          M java/engine/org/apache/derby/iapi/reference/DRDAConstants.java
          M java/client/org/apache/derby/client/net/DssConstants.java
          M java/client/org/apache/derby/client/net/Typdef.java

          Added DRDA and DB2 type codes and other protocol definitions for UDTs. Also added a manifest constant for the magic number which represents the maximum size of a DRDA buffer. The network layer is riddled with magic numbers. A wholesale cleanup is outside the scope of this patch.

          M java/engine/org/apache/derby/iapi/services/io/InputStreamUtil.java
          M java/engine/org/apache/derby/iapi/services/io/DynamicByteArrayOutputStream.java
          A java/shared/org/apache/derby/shared/common/io
          A java/shared/org/apache/derby/shared/common/io/DynamicByteArrayOutputStream.java
          A java/shared/org/apache/derby/shared/common/io/InputStreamUtil.java

          Moved some serialization support to the common area of the code tree so that the network client can use it.

          M java/engine/org/apache/derby/catalog/types/DecimalTypeIdImpl.java
          M java/engine/org/apache/derby/catalog/types/BaseTypeIdImpl.java

          These classes claimed to implement java.io.Serialization but did not fulfill the contract. This turned up during testing. The first class did not have a no-arg constructor as required by the contract. I believe that the contract for the second class was broken a long time ago when an effort was made to reduce the number of engine classes. As part of that effort, the base type ids had to be told what their format ids were after deserialization. That happens in a tricky piece of the formatable machinery which is not reproduced on the client side. I added logic to BaseTypeIdImpl so that it can reconstruct a format id based on the preserved type name.

          M java/drda/org/apache/derby/impl/drda/AppRequester.java
          M java/client/org/apache/derby/client/net/NetDatabaseMetaData.java
          M java/client/org/apache/derby/client/net/NetConnection.java

          Added methods to determine when both sides of the session support UDTs.

          M java/drda/org/apache/derby/impl/drda/FdocaConstants.java
          M java/drda/org/apache/derby/impl/drda/SQLTypes.java
          M java/drda/org/apache/derby/impl/drda/DDMWriter.java
          M java/drda/org/apache/derby/impl/drda/DRDAConnThread.java

          Server-side logic for exchanging UDT metadata and (de)serializing UDTs.

          M java/client/org/apache/derby/client/net/NetStatementRequest.java
          M java/client/org/apache/derby/client/net/NetStatementReply.java
          M java/client/org/apache/derby/client/net/Request.java
          M java/client/org/apache/derby/client/am/Cursor.java
          M java/client/org/apache/derby/client/am/PreparedStatement.java
          M java/client/org/apache/derby/client/am/Types.java
          M java/client/org/apache/derby/client/am/ColumnMetaData.java

          Client-side logic for exchanging UDT metadata and (de)serializing UDTs.

          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/UDTTest.java

          Added a db proc for testing output parameters of UDT type. This is not currently used. However, it disclosed DERBY-4499.

          M java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/DatabaseMetaDataTest.java

          Changed this test to account for the fact that ResultSetMetaData and DatabaseMetaData now agree when the client and server are both at Derby level 10.6 or higher.

          Show
          Rick Hillegas added a comment - Attaching derby-4491-01-ab-networkTransport.diff. This is a first rev of a patch to address this bug. This patch is not ready to commit yet: I need to write new regression tests to track the new behavior. However, the existing regression tests pass cleanly for me. This patch does the following: 1) Makes network JDBC behavior mimic the embedded JDBC behavior as described in the previous comment. The behaviors agree when the client and server are Derby code at level 10.6 or higher. Otherwise, the network behavior continues to be what it was in previous releases--that is, the buggy behavior described by this JIRA. 2) To satisfy the JDBC contract, this patch implements a Derby-only extension to DRDA at the point in the metadata exchange where we currently write a null SQLUDTGRP. The extended protocol is described in the header comment for DRDAConnThread.writeSQLUDTGRP(). 3) This patch adds DRDA and DB2 type codes for UDTs. The DRDA type codes are defined in the SQLUDTGRP section of the DRDA spec (Volume 1, section 5.6.4.10 SQL Descriptor User-Defined Type Group Description). Note that these DRDA type codes for UDTs do not appear in the summary 5-11 table in Volume 1--I believe that is an oversight on the part of the editors. I could not find DB2 type codes for UDTs. If someone knows what these are, that would be an improvement to this patch. In the meantime, I have invented type codes in a part of the code space which I think will not collide with DB2 evolution during our lifetimes. Since this is a Derby-only extension, I believe we are safe here regardless of how DB2 evolves. 4) Only small UDTs can be transported across DRDA right now. A small UDT cannot serialize to more than 32767 (0x7FFF) bytes, the maximum size of a DRDA network protocol buffer. In the future, we may want to add support for big UDTs. My feeling is that we will want to give the type designer some way to declare that a UDT is big--for instance, the UDT's Java class could implement a vacuous org.apache.derby.types.BigUDT interface. Based on that declaration, we could stream the big UDTs across the network. Based on this BigUDT declaration, we could also optimize Store and language support for these values and we could build support for SQL UDT locators. All of that work, however, requires community discussion and falls outside the scope of this JIRA. 5) In order to support UDT serialization across the network, two classes have been moved into the common part of the code tree so that the network client can use them: DynamicByteArrayOutputStream and InputStreamUtil. More work is needed: A) Regression and compatibility tests need to be added. B) DERBY-4449 needs to be fixed so that we can use UDTs as output parameters in db procs. Touches the following files: M java/engine/org/apache/derby/loc/messages.xml M java/shared/org/apache/derby/shared/common/reference/SQLState.java New error messages. M java/shared/org/apache/derby/shared/common/reference/JDBC30Translation.java M java/engine/org/apache/derby/iapi/types/DataTypeUtilities.java M java/engine/org/apache/derby/catalog/types/TypeDescriptorImpl.java Replaced some magic numbers with manifest constants. M java/engine/org/apache/derby/iapi/reference/DRDAConstants.java M java/client/org/apache/derby/client/net/DssConstants.java M java/client/org/apache/derby/client/net/Typdef.java Added DRDA and DB2 type codes and other protocol definitions for UDTs. Also added a manifest constant for the magic number which represents the maximum size of a DRDA buffer. The network layer is riddled with magic numbers. A wholesale cleanup is outside the scope of this patch. M java/engine/org/apache/derby/iapi/services/io/InputStreamUtil.java M java/engine/org/apache/derby/iapi/services/io/DynamicByteArrayOutputStream.java A java/shared/org/apache/derby/shared/common/io A java/shared/org/apache/derby/shared/common/io/DynamicByteArrayOutputStream.java A java/shared/org/apache/derby/shared/common/io/InputStreamUtil.java Moved some serialization support to the common area of the code tree so that the network client can use it. M java/engine/org/apache/derby/catalog/types/DecimalTypeIdImpl.java M java/engine/org/apache/derby/catalog/types/BaseTypeIdImpl.java These classes claimed to implement java.io.Serialization but did not fulfill the contract. This turned up during testing. The first class did not have a no-arg constructor as required by the contract. I believe that the contract for the second class was broken a long time ago when an effort was made to reduce the number of engine classes. As part of that effort, the base type ids had to be told what their format ids were after deserialization. That happens in a tricky piece of the formatable machinery which is not reproduced on the client side. I added logic to BaseTypeIdImpl so that it can reconstruct a format id based on the preserved type name. M java/drda/org/apache/derby/impl/drda/AppRequester.java M java/client/org/apache/derby/client/net/NetDatabaseMetaData.java M java/client/org/apache/derby/client/net/NetConnection.java Added methods to determine when both sides of the session support UDTs. M java/drda/org/apache/derby/impl/drda/FdocaConstants.java M java/drda/org/apache/derby/impl/drda/SQLTypes.java M java/drda/org/apache/derby/impl/drda/DDMWriter.java M java/drda/org/apache/derby/impl/drda/DRDAConnThread.java Server-side logic for exchanging UDT metadata and (de)serializing UDTs. M java/client/org/apache/derby/client/net/NetStatementRequest.java M java/client/org/apache/derby/client/net/NetStatementReply.java M java/client/org/apache/derby/client/net/Request.java M java/client/org/apache/derby/client/am/Cursor.java M java/client/org/apache/derby/client/am/PreparedStatement.java M java/client/org/apache/derby/client/am/Types.java M java/client/org/apache/derby/client/am/ColumnMetaData.java Client-side logic for exchanging UDT metadata and (de)serializing UDTs. M java/testing/org/apache/derbyTesting/functionTests/tests/lang/UDTTest.java Added a db proc for testing output parameters of UDT type. This is not currently used. However, it disclosed DERBY-4499 . M java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/DatabaseMetaDataTest.java Changed this test to account for the fact that ResultSetMetaData and DatabaseMetaData now agree when the client and server are both at Derby level 10.6 or higher.
          Hide
          Dag H. Wanvik added a comment -

          Thanks for addressing this nice extension to UDTs, Rick!. Just a quick question before I
          take a more detailed look.. what happens if the UDT is not in the client's CLASSPATH? Will it just be a plain Object when retrieved from JDBC?

          Show
          Dag H. Wanvik added a comment - Thanks for addressing this nice extension to UDTs, Rick!. Just a quick question before I take a more detailed look.. what happens if the UDT is not in the client's CLASSPATH? Will it just be a plain Object when retrieved from JDBC?
          Hide
          Rick Hillegas added a comment -

          Thanks for taking a look at this patch, Dag. If the UDT is not in the client's classpath, then I expect the following behavior:

          1) The metadata calls will still work. That is, DatabaseMetaData.getColumnClassName() and ParameterMetaData.getColumnClassName() will still return the name of the class bound to the UDT

          2) However, (de)serialization will fail on a class not found error.

          Show
          Rick Hillegas added a comment - Thanks for taking a look at this patch, Dag. If the UDT is not in the client's classpath, then I expect the following behavior: 1) The metadata calls will still work. That is, DatabaseMetaData.getColumnClassName() and ParameterMetaData.getColumnClassName() will still return the name of the class bound to the UDT 2) However, (de)serialization will fail on a class not found error.
          Hide
          Dag H. Wanvik added a comment -

          Thanks for clarifying that, Rick. We should probably mention this in the docs.

          Show
          Dag H. Wanvik added a comment - Thanks for clarifying that, Rick. We should probably mention this in the docs.
          Hide
          Rick Hillegas added a comment -

          Thanks, Dag. I agree. The functional spec doesn't talk about this behavior either. After this bug is fixed, I should update the functional spec and highlight this behavior for mentioning in the user docs:

          a) If you try to retrieve a UDT whose class isn't on your classpath, you will see a class not found exception

          b) If you try to set a UDT parameter with an object which isn't an instance of the UDT class, you will get a coercion error.

          Thanks.

          Show
          Rick Hillegas added a comment - Thanks, Dag. I agree. The functional spec doesn't talk about this behavior either. After this bug is fixed, I should update the functional spec and highlight this behavior for mentioning in the user docs: a) If you try to retrieve a UDT whose class isn't on your classpath, you will see a class not found exception b) If you try to set a UDT parameter with an object which isn't an instance of the UDT class, you will get a coercion error. Thanks.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-4491-01-ad-networkTransport .diff. This patch adds regression and compatibility tests to the previous patch and fixes a bug disclosed by those tests. I believe that this patch is commit-worthy.

          The compatibility tests verify that the new behavior only appears if both client and server are Derby code at level 10.6 or higher. In all other cases, the old behavior prevails. I ran the compatibility tests with the following combinations:

          o VMs used were 1.4, Java 5, and Java 6 for both clients and servers.

          o Clients ran at levels 10.0.2.1, 10.1.3.1, 10.2.2.1, 10.3.3.0, 10.4.2.1, 10.5.3.0, and 10.6.0.0 against a 10.6.0.0 server.

          o A 10.6.0.0 client ran against servers at all of the levels listed above.

          o As a sanity check that the compatibility tests still run cleanly against old releases, I also ran a 10.3.3.0 client against a 10.5.3.0 server and vice versa.

          Note that the JCC driver was tested because that is the driver used when the client jar is 10.0.2.1.

          One other note: with the current state of the trunk (without this patch), we get a protocol error when trying to use the trunk's NetworkServerControl to ping a server at various lower rev levels. I did not systematically map out the combinations that give rise to protocol errors. I don't consider this to be a serious defect and it may already be understood by our network experts. However, I had to disable the ping logic in the compatibility tests. With this patch, the tests simply wait for a while to give the server a chance to come up. If someone wants to fix this defect, then they can re-enable the ping while they are in there.

          Touches the following files in addition to the files touched by the previous patch:

          M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/CompatibilityCombinations.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/CompatibilitySuite.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/JDBCDriverTest.java
          M java/testing/org/apache/derbyTesting/functionTests/util/DerbyJUnitTest.java

          Show
          Rick Hillegas added a comment - Attaching derby-4491-01-ad-networkTransport .diff. This patch adds regression and compatibility tests to the previous patch and fixes a bug disclosed by those tests. I believe that this patch is commit-worthy. The compatibility tests verify that the new behavior only appears if both client and server are Derby code at level 10.6 or higher. In all other cases, the old behavior prevails. I ran the compatibility tests with the following combinations: o VMs used were 1.4, Java 5, and Java 6 for both clients and servers. o Clients ran at levels 10.0.2.1, 10.1.3.1, 10.2.2.1, 10.3.3.0, 10.4.2.1, 10.5.3.0, and 10.6.0.0 against a 10.6.0.0 server. o A 10.6.0.0 client ran against servers at all of the levels listed above. o As a sanity check that the compatibility tests still run cleanly against old releases, I also ran a 10.3.3.0 client against a 10.5.3.0 server and vice versa. Note that the JCC driver was tested because that is the driver used when the client jar is 10.0.2.1. One other note: with the current state of the trunk (without this patch), we get a protocol error when trying to use the trunk's NetworkServerControl to ping a server at various lower rev levels. I did not systematically map out the combinations that give rise to protocol errors. I don't consider this to be a serious defect and it may already be understood by our network experts. However, I had to disable the ping logic in the compatibility tests. With this patch, the tests simply wait for a while to give the server a chance to come up. If someone wants to fix this defect, then they can re-enable the ping while they are in there. Touches the following files in addition to the files touched by the previous patch: M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/CompatibilityCombinations.java M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/CompatibilitySuite.java M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/JDBCDriverTest.java M java/testing/org/apache/derbyTesting/functionTests/util/DerbyJUnitTest.java
          Hide
          Rick Hillegas added a comment -

          Committed derby-4491-01-ad-networkTransport .diff at subversion revision 899733.

          Show
          Rick Hillegas added a comment - Committed derby-4491-01-ad-networkTransport .diff at subversion revision 899733.
          Hide
          Rick Hillegas added a comment -

          Fixed a sealing violation which surfaced in AssertFailureTest when run against insane builds: subversion revision 899819. The sealing violation was caused by the use of the SanityManager which I introduced in Request.java. I don't know why the SanityManager has been included in the common arm of the codeline if it gives rise to sealing violations.

          While I was in there, I removed the two io classes which I had put in the common arm. It may be that similar sealing violations will plague them. I cloned them so that there are now separate client and server copies of InputStreamUtil.java and DynamicByteArrayOutputStream.java.

          I will log a separate issue to track this sealing violation issue. We may need to reconsider the presence of the SanityManager in the common arm of the codeline.

          Show
          Rick Hillegas added a comment - Fixed a sealing violation which surfaced in AssertFailureTest when run against insane builds: subversion revision 899819. The sealing violation was caused by the use of the SanityManager which I introduced in Request.java. I don't know why the SanityManager has been included in the common arm of the codeline if it gives rise to sealing violations. While I was in there, I removed the two io classes which I had put in the common arm. It may be that similar sealing violations will plague them. I cloned them so that there are now separate client and server copies of InputStreamUtil.java and DynamicByteArrayOutputStream.java. I will log a separate issue to track this sealing violation issue. We may need to reconsider the presence of the SanityManager in the common arm of the codeline.
          Hide
          Knut Anders Hatlen added a comment -

          The problems in AssertFailureTest may be caused by this code in client.net.Request:

          + final void writeUDT( Object val ) throws SqlException
          + {
          + // should not be called if val is null
          + if ( val == null )
          +

          { + SanityManager.THROWASSERT( "UDT is null" ); + }

          Since the SanityManager class is not included in insane jars, the call to THROWASSERT() must be guarded by "if (SanityManager.DEBUG)".

          Show
          Knut Anders Hatlen added a comment - The problems in AssertFailureTest may be caused by this code in client.net.Request: + final void writeUDT( Object val ) throws SqlException + { + // should not be called if val is null + if ( val == null ) + { + SanityManager.THROWASSERT( "UDT is null" ); + } Since the SanityManager class is not included in insane jars, the call to THROWASSERT() must be guarded by "if (SanityManager.DEBUG)".
          Hide
          Myrna van Lunteren added a comment -

          Since this check-in I see a test failure with ibm's j9: http://people.apache.org/~myrnavl/derby_test_results/main/windows/testlog/weme6.2/899875-suites.All_diff.txt

          1) test_10_parameterMetaData(org.apache.derbyTesting.functionTests.tests.lang.UDTTest)java.lang.NoSuchMethodError: java/sql/PreparedStatement.getParameterMetaData()Ljava/sql/ParameterMetaData;
          at org.apache.derbyTesting.functionTests.tests.lang.UDTTest.checkPMD(UDTTest.java:834)
          at org.apache.derbyTesting.functionTests.tests.lang.UDTTest.test_10_parameterMetaData(UDTTest.java:702)
          at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:195)
          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)

          It seems to me this must be related to the recent changes to this test...

          Show
          Myrna van Lunteren added a comment - Since this check-in I see a test failure with ibm's j9: http://people.apache.org/~myrnavl/derby_test_results/main/windows/testlog/weme6.2/899875-suites.All_diff.txt 1) test_10_parameterMetaData(org.apache.derbyTesting.functionTests.tests.lang.UDTTest)java.lang.NoSuchMethodError: java/sql/PreparedStatement.getParameterMetaData()Ljava/sql/ParameterMetaData; at org.apache.derbyTesting.functionTests.tests.lang.UDTTest.checkPMD(UDTTest.java:834) at org.apache.derbyTesting.functionTests.tests.lang.UDTTest.test_10_parameterMetaData(UDTTest.java:702) at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:195) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) It seems to me this must be related to the recent changes to this test...
          Hide
          Rick Hillegas added a comment -

          Thanks for finding this error, Myrna. It occurs because PreparedStatement.getParameterMetaData() is not available on JSR 169 platforms. I have disabled that test case on those vms with subversion revision 901277.

          Show
          Rick Hillegas added a comment - Thanks for finding this error, Myrna. It occurs because PreparedStatement.getParameterMetaData() is not available on JSR 169 platforms. I have disabled that test case on those vms with subversion revision 901277.
          Hide
          Rick Hillegas added a comment -

          Brackted an assertion in DDMWriter with a SanityManager.DEBUG check: subversion revision 901420.

          Show
          Rick Hillegas added a comment - Brackted an assertion in DDMWriter with a SanityManager.DEBUG check: subversion revision 901420.
          Hide
          Kristian Waagan added a comment -

          Reopening because it looks like NetDatabaseMetadata.serverSupportsUDTs returns the wrong value (see patch derby-4491-02-aa-supportsUDTs.diff).
          I plan to commit this patch shortly, and I will also set the fix version of this issue to 10.6.

          Show
          Kristian Waagan added a comment - Reopening because it looks like NetDatabaseMetadata.serverSupportsUDTs returns the wrong value (see patch derby-4491-02-aa-supportsUDTs.diff). I plan to commit this patch shortly, and I will also set the fix version of this issue to 10.6.
          Hide
          Rick Hillegas added a comment -

          Thanks for catching this, Kristian. This may indicate that there is a missing compatibility test case. I will re-run the compatibility tests after you check in this fix.

          Show
          Rick Hillegas added a comment - Thanks for catching this, Kristian. This may indicate that there is a missing compatibility test case. I will re-run the compatibility tests after you check in this fix.
          Hide
          Kristian Waagan added a comment -

          Thanks for the quick feedback, Rick. You're probably right about the missing compatibility test case.
          I committed patch 02-aa to trunk with revision 918112.

          I'll leave it up to you to resolve / close the issue. In case you add a new test case, I'll be interested in having a look at your code for another issue I'm working on

          Show
          Kristian Waagan added a comment - Thanks for the quick feedback, Rick. You're probably right about the missing compatibility test case. I committed patch 02-aa to trunk with revision 918112. I'll leave it up to you to resolve / close the issue. In case you add a new test case, I'll be interested in having a look at your code for another issue I'm working on
          Hide
          Rick Hillegas added a comment -

          I have re-run the compatibility tests using combinations of clients and servers at trunk and 10.5.3.0 levels. This should stress the change made by Kristian since a 10.5.3.0 server supports data caching but not UDTs. The compatibility tests passed cleanly in this combination.

          I believe that the tests previously passed because the only check for serverSupportsUDT() occurs in NetStatementReply.parseSQLUDTGRP():

          if ( !(jdbcType == Types.JAVA_OBJECT) || !netAgent_.netConnection_.serverSupportsUDTs() )

          This condition would have been satisified even if serverSupportsUDTs() returned the wrong value because if the server really wasn't at level 10.6, then jdbcType would be Types.LONGVARBINARY.

          I suppose this means that we could remove the serverSupportsUDTs() method. However, I recommend keeping this redundant sanity check because:

          a) it causes no problems

          b) it flags the point in a code where we need to be UDT-aware

          Show
          Rick Hillegas added a comment - I have re-run the compatibility tests using combinations of clients and servers at trunk and 10.5.3.0 levels. This should stress the change made by Kristian since a 10.5.3.0 server supports data caching but not UDTs. The compatibility tests passed cleanly in this combination. I believe that the tests previously passed because the only check for serverSupportsUDT() occurs in NetStatementReply.parseSQLUDTGRP(): if ( !(jdbcType == Types.JAVA_OBJECT) || !netAgent_.netConnection_.serverSupportsUDTs() ) This condition would have been satisified even if serverSupportsUDTs() returned the wrong value because if the server really wasn't at level 10.6, then jdbcType would be Types.LONGVARBINARY. I suppose this means that we could remove the serverSupportsUDTs() method. However, I recommend keeping this redundant sanity check because: a) it causes no problems b) it flags the point in a code where we need to be UDT-aware
          Hide
          Kathey Marsden added a comment -

          reopen to add derby_backport_reject label

          Show
          Kathey Marsden added a comment - reopen to add derby_backport_reject label

            People

            • Assignee:
              Rick Hillegas
              Reporter:
              Rick Hillegas
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development