Derby
  1. Derby
  2. DERBY-959

Allow use of DRDA QRYDTA block sizes greater than 32K

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 10.1.1.0, 10.1.2.1, 10.2.1.6
    • Fix Version/s: 10.2.1.6
    • Labels:
      None
    • Bug behavior facts:
      Performance

      Description

      Currently, the Network Server and Network Client use a 32K blocksize when
      returning database data in QRYDTA blocks.

      I came across the following statement in the DRDA spec (Volume 1, page 12):

      Blocking applies only to the QRYDTA reply objects. Each
      query block is a QRYDTA DSS. The maximum query block size
      value allowed in the qryblksz parameter is increased from
      32K to 10M, thus accomodating the larger data volumes
      required by modern, more data-intensive applications.

      The importance of larger block sizes depends strong on the application
      profile.. For example, many applications perform single-record selects,
      and they are not influenced by the block size.

      But for some applications, it seems like the ability to use a larger block
      size could be quite valuable.

      1. SmallRows_32kBlkSz.trace
        395 kB
        Sunitha Kambhampati
      2. SmallRows_64kBlkSz.trace
        394 kB
        Sunitha Kambhampati
      3. odbc_rowgreaterthan64k.trace
        1.44 MB
        Sunitha Kambhampati
      4. derby959.review.stat.txt
        0.4 kB
        Sunitha Kambhampati
      5. 959Notes_v1.html
        17 kB
        Sunitha Kambhampati
      6. derby959.review.diff.txt
        6 kB
        Sunitha Kambhampati

        Issue Links

          Activity

          Hide
          Bryan Pendleton added a comment -

          Thanks Sunitha for addressing this! Fix verified as complete.

          Show
          Bryan Pendleton added a comment - Thanks Sunitha for addressing this! Fix verified as complete.
          Hide
          Sunitha Kambhampati added a comment -

          This fix has been code for sometime. I have verified this fix with the odbc client driver.

          Show
          Sunitha Kambhampati added a comment - This fix has been code for sometime. I have verified this fix with the odbc client driver.
          Hide
          Mike Matrigali added a comment -

          I missed one file new file in previous commit, here is the svn commit for it:

          m1_142:155>svn commit

          Adding java\testing\org\apache\derbyTesting\functionTests\tests\derbynet\values64kblksz.inc
          Transmitting file data .
          Committed revision 416486.

          Show
          Mike Matrigali added a comment - I missed one file new file in previous commit, here is the svn commit for it: m1_142:155>svn commit Adding java\testing\org\apache\derbyTesting\functionTests\tests\derbynet\values64kblksz.inc Transmitting file data . Committed revision 416486.
          Hide
          Mike Matrigali added a comment -

          I committed the xxx patch to the trunk:
          m1_142:151>svn commit

          Sending java\drda\org\apache\derby\impl\drda\CodePoint.java
          Sending java\testing\org\apache\derbyTesting\functionTests\master\testProtocol.out
          Sending java\testing\org\apache\derbyTesting\functionTests\tests\derbynet\protocol.tests
          Sending java\testing\org\apache\derbyTesting\functionTests\tests\derbynet\testProtocol_app.properties
          Transmitting file data ....
          Committed revision 416485.

          Show
          Mike Matrigali added a comment - I committed the xxx patch to the trunk: m1_142:151>svn commit Sending java\drda\org\apache\derby\impl\drda\CodePoint.java Sending java\testing\org\apache\derbyTesting\functionTests\master\testProtocol.out Sending java\testing\org\apache\derbyTesting\functionTests\tests\derbynet\protocol.tests Sending java\testing\org\apache\derbyTesting\functionTests\tests\derbynet\testProtocol_app.properties Transmitting file data .... Committed revision 416485.
          Hide
          Sunitha Kambhampati added a comment -

          Discussion happened on this issue on derby-dev. Here is the link to the discussion that happened on derby-dev
          http://www.nabble.com/-jira--Created%3A-%28DERBY-959%29-Allow-use-of-DRDA-QRYDTA-block-sizes-greater-than-32K-t1106974.html#a2892273

          I have opened two other jira issues for the tasks related to this issue. .
          DERBY-1441 - client side changes to support query block size greater than 32k
          DERBY-1442- to do performance analysis and come up with a 'good' query block size value for the client to send to the server.

          Show
          Sunitha Kambhampati added a comment - Discussion happened on this issue on derby-dev. Here is the link to the discussion that happened on derby-dev http://www.nabble.com/-jira--Created%3A-%28DERBY-959%29-Allow-use-of-DRDA-QRYDTA-block-sizes-greater-than-32K-t1106974.html#a2892273 I have opened two other jira issues for the tasks related to this issue. . DERBY-1441 - client side changes to support query block size greater than 32k DERBY-1442 - to do performance analysis and come up with a 'good' query block size value for the client to send to the server.
          Hide
          Sunitha Kambhampati added a comment -

          The new C client that I was testing the server with is sending a qryblksz of 65535 and I gathered server traces with it. I am attaching the trace files here .

          odbc_rowgreaterthan64k.trace – using C client, trace has select of a table with 4 rows each row having more than 64k of data. The table has 341 columns of type char(200).

          SmallRows_32kBlkSz.trace - this server trace was gathered when running with the client driver(jdbc). query is a simple select * from tbl. where the table has rows ~ 10k each.

          SmallRows_64kBlkSz.trace - this server trace was gathered when running with the C client that sends a qryblksz of 65535. query is a simple select * from tbl. where the table has rows ~ 10k each.

          Please let me know if you need more information to understand these traces. Thanks.

          Show
          Sunitha Kambhampati added a comment - The new C client that I was testing the server with is sending a qryblksz of 65535 and I gathered server traces with it. I am attaching the trace files here . odbc_rowgreaterthan64k.trace – using C client, trace has select of a table with 4 rows each row having more than 64k of data. The table has 341 columns of type char(200). SmallRows_32kBlkSz.trace - this server trace was gathered when running with the client driver(jdbc). query is a simple select * from tbl. where the table has rows ~ 10k each. SmallRows_64kBlkSz.trace - this server trace was gathered when running with the C client that sends a qryblksz of 65535. query is a simple select * from tbl. where the table has rows ~ 10k each. Please let me know if you need more information to understand these traces. Thanks.
          Hide
          Sunitha Kambhampati added a comment -

          I am attaching a patch 'derby959.review.diff.txt' to get feedback on the server changes to allow for query block size greater than 32k. From the test cases that I tried, it seems to me that the server already knows how to handle sending QRYDTA greater than 32k.
          Please see the attached html file 959Notes_v1.html for more notes:

          This is a network server only change and I ran the derbynetclientmats and derbynetmats with 1.4.2 sun jvm OK on windows.

          I'd appreciate your comments and feedback.

          Thanks.

          Show
          Sunitha Kambhampati added a comment - I am attaching a patch 'derby959.review.diff.txt' to get feedback on the server changes to allow for query block size greater than 32k. From the test cases that I tried, it seems to me that the server already knows how to handle sending QRYDTA greater than 32k. Please see the attached html file 959Notes_v1.html for more notes: This is a network server only change and I ran the derbynetclientmats and derbynetmats with 1.4.2 sun jvm OK on windows. I'd appreciate your comments and feedback. Thanks.

            People

            • Assignee:
              Sunitha Kambhampati
              Reporter:
              Bryan Pendleton
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development