[DERBY-6888] Large User-Defined Types break Network Client

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 10.11.1.1, 10.12.1.1
• Bug behavior facts: Embedded/Client difference, Seen in production
• Assignee: Unassigned
• Reporter: Steven Schaefer
• Created: 03/May/16 13:40
• Updated: 04/May/16 02:03

Description

In the project I work on at Nexidia we have been working with a derby in probably an unusual way where we make use of both the embedded and network clients.

We also make some use of user-defined types and have run into some inconsistencies between those clients.

Occasionally some of our user-defined types are above 32k. This is ok for the embedded client; it seems to read and write those without issue. However, the network client has some serious trouble.

1) Writes: The network client doesn't allow writing UDTs above 32k. I get this helpful error message (more on this at the end...) "java.sql.SQLDataException: The resulting value is outside the range for the data type 32767." Stack trace ends up pointing to org.apache.derby.client.net.Request.writeUDT, and its code shows a hardcoded check.

2) Reads: Since our (multi-server) system runs with both clients for different needs, our embedded client helpfully inserts UDTs greater than 32k. The network client doesn't appreciate this much. Reading one of these large UDTs results in the following: "java.sql.SQLNonTransientConnectionException: Insufficient data while reading from the network - expected a minimum of 6 bytes and received only 0 bytes. The connection has been terminated".

Even better, the next time we try to create a new connection, it fails with "java.sql.SQLNonTransientConnectionException: A network protocol error was encountered and the connection has been terminated: A protocol error (Data Stream Syntax Error) was detected. Reason: 0x19. Perhaps this is an attempt to open a plaintext connection to an SSL enabled server."

3) With debug jars, #1's error message format fails an assertion check:

org.apache.derby.shared.common.sanity.AssertFailure: ASSERT FAILED Number of parameters expected for message id 22003 (1) does not match number of arguments received (2)
at org.apache.derby.shared.common.sanity.SanityManager.ASSERT(SanityManager.java:120)
at org.apache.derby.shared.common.i18n.MessageUtil.formatMessage(MessageUtil.java:209)
at org.apache.derby.shared.common.i18n.MessageUtil.getCompleteMessage(MessageUtil.java:118)
at org.apache.derby.shared.common.i18n.MessageUtil.getCompleteMessage(MessageUtil.java:158)
at org.apache.derby.shared.common.i18n.MessageUtil.getCompleteMessage(MessageUtil.java:74)
at org.apache.derby.client.am.SqlException.<init>(SqlException.java:169)
at org.apache.derby.client.am.SqlException.<init>(SqlException.java:203)
at org.apache.derby.client.net.Request.writeUDT(Request.java:1418)

Expected

I expect some consistency here between the two clients. Either both accept UDTs greater than 32k, or both reject them. Further, if a limit of 32k is still intended, it'd be great of documentation was a bit loud about that limitation. DERBY-4491 comments indicate some sort of "large UDT" may be appropriate, but it's not immediately clear to me if that's still needed. And of course I expect subsequent connections to succeed! I'm also a little surprised large blobs seem to work ok in this scenario, but I'm hoping that's an indication UDTs shouldn't be limited to 32k too.

We've been able to refactor out the network client to workaround the issue, but it's not yet clear if this is an ideal long-term solution for us.

Reproducing:

I have attached several sample programs:

• EmbeddedTest: Inserts & Selects a blob and UDT that's 50k. I expect this program to run without issue.
• NetworkWriteTest: Attempts to insert a blob and UDT that's 50k. This will demonstrate blobs are ok here, but #1 for UDTs (or #3 with debug jars).
• NetworkLargeUDTReadTest: This demonstrates #2. It will attempt to read the large UDT written by EmbeddedTest. Finally, it will then try to create a new connection demonstrating the 0x19 error.

To run:

• unzip, then cd udt_network_repo
• gradlew build
• java -cp "build\libs*" com.nexidia.udtrepro.EmbeddedTest [preferred derby home]
• java -cp "build\libs*" com.nexidia.udtrepro.NetworkWriteTest [preferred derby home]
• java -cp "build\libs*" com.nexidia.udtrepro.NetworkLargeUDTReadTest [preferred derby home]

Note: EmbeddedTest should be run before NetworkLargeUDTReadTest

I have reproed all with 10.12 as well. All tests run in Windows 7 w/ Java 8 u77.

Issue Links

• is related to: DERBY-4491 The network client changes UDTs into Strings and returns their type as LONGVARBINARY.