Attaching derby-4491-01-ab-networkTransport.diff. This is a first rev of a patch to address this bug. This patch is not ready to commit yet: I need to write new regression tests to track the new behavior. However, the existing regression tests pass cleanly for me.
This patch does the following:
1) Makes network JDBC behavior mimic the embedded JDBC behavior as described in the previous comment. The behaviors agree when the client and server are Derby code at level 10.6 or higher. Otherwise, the network behavior continues to be what it was in previous releases--that is, the buggy behavior described by this JIRA.
2) To satisfy the JDBC contract, this patch implements a Derby-only extension to DRDA at the point in the metadata exchange where we currently write a null SQLUDTGRP. The extended protocol is described in the header comment for DRDAConnThread.writeSQLUDTGRP().
3) This patch adds DRDA and DB2 type codes for UDTs. The DRDA type codes are defined in the SQLUDTGRP section of the DRDA spec (Volume 1, section 18.104.22.168 SQL Descriptor User-Defined Type Group Description). Note that these DRDA type codes for UDTs do not appear in the summary 5-11 table in Volume 1--I believe that is an oversight on the part of the editors. I could not find DB2 type codes for UDTs. If someone knows what these are, that would be an improvement to this patch. In the meantime, I have invented type codes in a part of the code space which I think will not collide with DB2 evolution during our lifetimes. Since this is a Derby-only extension, I believe we are safe here regardless of how DB2 evolves.
4) Only small UDTs can be transported across DRDA right now. A small UDT cannot serialize to more than 32767 (0x7FFF) bytes, the maximum size of a DRDA network protocol buffer. In the future, we may want to add support for big UDTs. My feeling is that we will want to give the type designer some way to declare that a UDT is big--for instance, the UDT's Java class could implement a vacuous org.apache.derby.types.BigUDT interface. Based on that declaration, we could stream the big UDTs across the network. Based on this BigUDT declaration, we could also optimize Store and language support for these values and we could build support for SQL UDT locators. All of that work, however, requires community discussion and falls outside the scope of this JIRA.
5) In order to support UDT serialization across the network, two classes have been moved into the common part of the code tree so that the network client can use them: DynamicByteArrayOutputStream and InputStreamUtil.
More work is needed:
A) Regression and compatibility tests need to be added.
DERBY-4449 needs to be fixed so that we can use UDTs as output parameters in db procs.
Touches the following files:
New error messages.
Replaced some magic numbers with manifest constants.
Added DRDA and DB2 type codes and other protocol definitions for UDTs. Also added a manifest constant for the magic number which represents the maximum size of a DRDA buffer. The network layer is riddled with magic numbers. A wholesale cleanup is outside the scope of this patch.
Moved some serialization support to the common area of the code tree so that the network client can use it.
These classes claimed to implement java.io.Serialization but did not fulfill the contract. This turned up during testing. The first class did not have a no-arg constructor as required by the contract. I believe that the contract for the second class was broken a long time ago when an effort was made to reduce the number of engine classes. As part of that effort, the base type ids had to be told what their format ids were after deserialization. That happens in a tricky piece of the formatable machinery which is not reproduced on the client side. I added logic to BaseTypeIdImpl so that it can reconstruct a format id based on the preserved type name.
Added methods to determine when both sides of the session support UDTs.
Server-side logic for exchanging UDT metadata and (de)serializing UDTs.
Client-side logic for exchanging UDT metadata and (de)serializing UDTs.
Added a db proc for testing output parameters of UDT type. This is not currently used. However, it disclosed
Changed this test to account for the fact that ResultSetMetaData and DatabaseMetaData now agree when the client and server are both at Derby level 10.6 or higher.