Add code to support request and return of locators over DRDA

• Key: DERBY-2347
• Type: Sub-task
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Øystein Grøvlen
• Reporter: Øystein Grøvlen
• Component/s: Network Client, Network Server
• Fix Version/s: 10.3.1.4

Description

- Make the client able to send DRDA OUTOVR commands to request locators from
  Network Server.
- Make the network server send locators instead of LOB values if locators has been
  requested.

This JIRA will not activate the code that request locators since the client is not yet able to handle locators.

Comments

[Øystein Grøvlen]

This work will be dependent on the Locator to LOB mapping that is part of DERBY-2257 .

[Øystein Grøvlen]

The attached patch, DERBY-2347.diff, adds the necessary code for the
client to request locators to be sent instead of LOB values, and the
corresponding code for the network server to handle such requests.
This is achieved by using the DRDA OUTOVR (Output Override) command
with the the first CNTQRY command for a query. It turned out that
much of the necessary code for generating and handling OUTOVR was
already there, and only few missing pieces were needed to get this to
work.

For now, the client will not request the use of locators since it is
not yet able to handle locators. (This will be added in (several)
later patches.)

The DRDA spec also describes an outovropt parameter to the OPNQRY
command. This can be used to specify whether an OUTOVR command will
only be sent with the first CNTQRY for the query, or it can be sent
with any CNTQRY. As stated above, this implementation will only send
OUTOVR with the first CNTQRY, and since DRDA specifies this as the
default behavior, I have not used the outovropt parameter. There is
currently nothing in the network server that rely on OUTOVR only being
used with the first CNTQRY, and no error is returned should later
CNTQRY countain OUTOVR.

Note that this patch is dependent on the patch for DERBY-2257 and
should not be committed until that is committed.I have run the junit
and derbyall test suites with this patch and the patch for DERBY-2257,
and no new errors was seen.

A more detailed explanation of the changes:

M java/drda/org/apache/derby/impl/drda/DRDAResultSet.java

       If the client has requested an output type to be overridden,
       return the requested type.

M java/drda/org/apache/derby/impl/drda/DRDAConnThread.java

       If the output type is a locator type, add the LOB reference to
       the locator-to-LOB-mapping of DERBY-2257, and write the
       returned locator as an integer.

M java/client/org/apache/derby/client/net/CodePointNameTable.java

       Add tracing text for the DRDA OUTOVR command. (This is for
       client, already existed in the server version of this file.)

M java/client/org/apache/derby/client/net/NetResultSetRequest.java

       Replaced the no-op code of buildOUTOVR() with the code to
       actually build the command. Only build it on the first CNTQRY
       command. This code will remain disabled until the necessary
       changes is made to make the client able to handle locators.

M java/client/org/apache/derby/client/net/NetStatementRequest.java

       In OUTOVR commands, the client will request locators for all
       LOB columns. 4 bytes will be used for locators.

[Kristian Waagan]

Removed trailing spaces from and refreshed the patch (attached 'DERBY-2347-v1.diff').
I looked at the changes and they look reasonable.
Even though the patch is small, another review from someone that knows DRDA stuff might be useful.

Note that the issue is still blocked and this patch must not be committed yet.

[Bryan Pendleton]

Why is the request to use locator format set on the CNTQRY rather than on the OPENQRY? I'm concerned about the DERBY-822 pre-fetching behavior; isn't it necessary to send output format behavior requests like these on the OPENQRY message so that they are honored even if the server does pre-fetching?

[Knut Anders Hatlen]

I don't think the DRDA protocol allows pre-fetching when the result contains LOBs. I haven't checked what the spec says, but I found these comments in DRDAConnThread:

processCommands():
if (stmt.getQryprctyp() == CodePoint.LMTBLKPRC &&
stmt.getQryrowset() != 0) {
// The DRDA spec allows us to send
// QRYDTA here if there are no LOB
// columns.

parseEXCSQLSTT():
/* Currently, if LMTBLKPRC is used, a pre-condition is that no lob columns.
* But in the future, when we do support LOB in LMTBLKPRC, the drda spec still
* does not allow LOB to be sent with OPNQRYRM. So this "if" here will have
* to add "no lob columns".
*/
if (stmt.getQryprctyp() == CodePoint.LMTBLKPRC)
writeQRYDTA(stmt);

[Øystein Grøvlen]

As far as I can see, the DRDA protocol does not provide any ways to request locators as part of the OPNQRY.

Anyhow, the client will be made so that it also handles receiving LOBs when it had requested locators. Hence, should pre-fetching include LOBs, which I do not think it does, it should still work.

If LOBs where small, it would be desireable to allow prefetching. However, if LOBs are small, you ideally would want the LOB values, not locators. Since the client will handle LOB values, anyway, I have been thinking about sending values instead of locators if LOBs are small, but I suspect that would be a violation of the DRDA protocol.

[Bryan Pendleton]

Thanks Knut Anders and Oystein. That makes complete sense. If you haven't already done so, it might be worth updating the comment that Knut Anders referenced to explain this nuance of locator handling as part of your patch.

[Øystein Grøvlen]

Since DErBY-2257 has now been committed, this patch is ready for commit.

[Kristian Waagan]

I'm running tests on this patch, and will commit late today or tomorrow.

I did not quite understand the request for an update of the comment(s) referenced, so I think more info is required for that update to happen.

[Kristian Waagan]

Committed patch to trunk with revision 515563.

Ran derbyall and suites.All with Sun JDK 5.0. Only setTransactionIsolationLevel failed (which I believe has been fixed by now).