Derby
  1. Derby
  2. DERBY-499

Expose BOOLEAN datatype to end users

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.1.1.0
    • Fix Version/s: 10.7.1.1
    • Component/s: SQL
    • Labels:
      None

      Description

      Veaceslav Chicu started an email thread on 8 August 2005 titled "boolean type". He was disappointed that Derby doesn't support the ansi BOOLEAN datatype. On closer inspection, Derby does internally support this type but does not expose this support to end users.

      Derby should let users declare table columns of type BOOLEAN. This should be an indexable datatype.

      1. jdk131BooleanFailures.zip
        67 kB
        Kathey Marsden
      2. bug499.diff
        254 kB
        Rick Hillegas
      3. bug499_rev4.diff
        254 kB
        Rick Hillegas
      4. bug499_rev3.diff
        254 kB
        Rick Hillegas
      5. bug499_rev2.diff
        254 kB
        Rick Hillegas
      6. bug499_jdk13tests.diff
        697 kB
        Rick Hillegas
      7. bug499_jdk13tests_rev2.diff
        693 kB
        Rick Hillegas
      8. bug499_doc.zip
        583 kB
        Rick Hillegas
      9. BooleanType.sxw
        12 kB
        Rick Hillegas
      10. BooleanType.sxw
        12 kB
        Rick Hillegas
      11. BooleanType.html
        30 kB
        Rick Hillegas
      12. BooleanType.html
        30 kB
        Rick Hillegas
      13. BooleanFS.html
        8 kB
        Rick Hillegas
      14. BooleanFS.html
        81 kB
        Rick Hillegas

        Issue Links

          Activity

          Hide
          Rick Hillegas added a comment -

          Closing this issue now that the user documentation is checked in.

          Show
          Rick Hillegas added a comment - Closing this issue now that the user documentation is checked in.
          Hide
          Rami Ojares added a comment -

          I will try my sword first with adding comments to derby (DERBY-4379)
          If that seems simple enough I will start helping you on this one.

          Show
          Rami Ojares added a comment - I will try my sword first with adding comments to derby ( DERBY-4379 ) If that seems simple enough I will start helping you on this one.
          Hide
          Rick Hillegas added a comment -

          Hi Rami,

          The problems with re-enabling the BOOLEAN datatype center on backward-compatibility concerns. I hope to be able to put some time during the 10.6 cycle into fixing DERBY-887. That's a casting bug which stood in the way of the first attempt to re-enable BOOLEAN. One way forward on this issue might be to split the work into smaller increments. For instance, if we just implemented BOOLEAN for embedded usage, that might reduce the compatibility issues. I would be happy to give you advice about how to tackle this issue if you want to help out.

          I stopped working on this issue because it was not a high priority for my company.

          Show
          Rick Hillegas added a comment - Hi Rami, The problems with re-enabling the BOOLEAN datatype center on backward-compatibility concerns. I hope to be able to put some time during the 10.6 cycle into fixing DERBY-887 . That's a casting bug which stood in the way of the first attempt to re-enable BOOLEAN. One way forward on this issue might be to split the work into smaller increments. For instance, if we just implemented BOOLEAN for embedded usage, that might reduce the compatibility issues. I would be happy to give you advice about how to tackle this issue if you want to help out. I stopped working on this issue because it was not a high priority for my company.
          Hide
          Rami Ojares added a comment -

          Isn't it about time this issue is picked up?
          I mean boolean type is the simplest and most elementary of all datatypes.
          It seems quite amazing that this type would be too hard to implement whereas Derby has such obscure and useless types like XML?

          I mean it almost seems the world has gone mad!

          But seriously, why did you abandon the implementation Rick?

          I would be happy to help when time permits.

          Show
          Rami Ojares added a comment - Isn't it about time this issue is picked up? I mean boolean type is the simplest and most elementary of all datatypes. It seems quite amazing that this type would be too hard to implement whereas Derby has such obscure and useless types like XML? I mean it almost seems the world has gone mad! But seriously, why did you abandon the implementation Rick? I would be happy to help when time permits.
          Hide
          Rick Hillegas added a comment -

          Thanks for the interest in this feature, Laurent. The issue is currently unassigned, waiting for someone to pick it up. I would be happy to coach someone through the process.

          Show
          Rick Hillegas added a comment - Thanks for the interest in this feature, Laurent. The issue is currently unassigned, waiting for someone to pick it up. I would be happy to coach someone through the process.
          Hide
          Laurent ROCHE added a comment -

          This spec looks good to me !

          When can we expect to have a BOOLEAN data type in Derby ?

          Show
          Laurent ROCHE added a comment - This spec looks good to me ! When can we expect to have a BOOLEAN data type in Derby ?
          Hide
          Rick Hillegas added a comment -

          I abandoned a previous attempt to implement the BOOLEAN datatype. Now I am attaching a new functional spec, describing a new attempt. Your feedback is appreciated.

          Show
          Rick Hillegas added a comment - I abandoned a previous attempt to implement the BOOLEAN datatype. Now I am attaching a new functional spec, describing a new attempt. Your feedback is appreciated.
          Hide
          Andrew McIntyre added a comment -

          Unsetting Fix Version on unassigned issues.

          Show
          Andrew McIntyre added a comment - Unsetting Fix Version on unassigned issues.
          Hide
          Rick Hillegas added a comment -

          Attaching an html version of the DRDA proposal for people who don't have OpenOffice.

          Show
          Rick Hillegas added a comment - Attaching an html version of the DRDA proposal for people who don't have OpenOffice.
          Hide
          Rick Hillegas added a comment -

          Attaching a slightly revised version of the DRDA proposal for BOOLEAN. This revision includes some copy-editting changes. In addition, it mentions coercion between BOOLEAN and ONE-BYTE-INTEGER rather than TWO-BYTE-INTEGER. The DBIOP Consortium is now ready to entertain proposals from its advisers. This is the version I will submit.

          Show
          Rick Hillegas added a comment - Attaching a slightly revised version of the DRDA proposal for BOOLEAN. This revision includes some copy-editting changes. In addition, it mentions coercion between BOOLEAN and ONE-BYTE-INTEGER rather than TWO-BYTE-INTEGER. The DBIOP Consortium is now ready to entertain proposals from its advisers. This is the version I will submit.
          Hide
          Rick Hillegas added a comment -

          Assign to release 11 because of the compatibility issues surrounding the current, non-ANSI casting behavior of BOOLEAN.

          Show
          Rick Hillegas added a comment - Assign to release 11 because of the compatibility issues surrounding the current, non-ANSI casting behavior of BOOLEAN.
          Hide
          Rick Hillegas added a comment -

          Unassigning myself from this issue. If someone wants to tackle this in the future, please consult DERBY-1029. In order to implement BOOLEAN, you will need to:

          1) Get the BOOLEAN datatype into the DRDA spec
          2) Secure the community's permission for a major release which will let you change the behavior of implicit and explicit BOOLEAN casts.

          Show
          Rick Hillegas added a comment - Unassigning myself from this issue. If someone wants to tackle this in the future, please consult DERBY-1029 . In order to implement BOOLEAN, you will need to: 1) Get the BOOLEAN datatype into the DRDA spec 2) Secure the community's permission for a major release which will let you change the behavior of implicit and explicit BOOLEAN casts.
          Hide
          Rick Hillegas added a comment -

          Linking this issue to DERBY-1029. That issue backs out most of the work done for DERBY-499.

          Show
          Rick Hillegas added a comment - Linking this issue to DERBY-1029 . That issue backs out most of the work done for DERBY-499 .
          Hide
          Rick Hillegas added a comment -

          I am also attaching the DRDA proposal as html. The two versions are BooleanType.sxw and BooleanType.html.

          Show
          Rick Hillegas added a comment - I am also attaching the DRDA proposal as html. The two versions are BooleanType.sxw and BooleanType.html.
          Hide
          Rick Hillegas added a comment -

          I am attaching a proposal for adding the BOOLEAN datatype to the DRDA spec.

          Show
          Rick Hillegas added a comment - I am attaching a proposal for adding the BOOLEAN datatype to the DRDA spec.
          Hide
          Kathey Marsden added a comment -

          Apparently there was a problem with my original mail going to the list so I will make a link here.
          http://issues.apache.org/jira/browse/DERBY-499#action_12366383

          In Ricks response he said

          >Let me punch up the importance of this point: Our own production code already >relies on pre-existing illegal casts and comparisons. Quite likely, some of our customers do too.

          Yes, this is one problem I am happy to defer to another day. It would only be exacerbated now by allowing user-declarable BOOLEAN columns/args.

          >In summary:

          >o I think that we should back out user-declarable BOOLEAN columns/args one way or another.

          great in svn merge you can just reverse the order of change number to back out the patch, but there may be conflicts due to intervening changes.

          svn merge -r 1234:1233 https://svn.apache.org/repos/asf/db/derby/code/trunk

          >o If (1) and (2) are showstoppers, they are showstoppers regardless of what we do about user-declarable BOOLEAN columns/args.

          I don't think so. If we just leave it just like 10.1 is for now and think on this whole problem a bit more for how to create a BOOLEAN type for 10.3.

          Show
          Kathey Marsden added a comment - Apparently there was a problem with my original mail going to the list so I will make a link here. http://issues.apache.org/jira/browse/DERBY-499#action_12366383 In Ricks response he said >Let me punch up the importance of this point: Our own production code already >relies on pre-existing illegal casts and comparisons. Quite likely, some of our customers do too. Yes, this is one problem I am happy to defer to another day. It would only be exacerbated now by allowing user-declarable BOOLEAN columns/args. >In summary: >o I think that we should back out user-declarable BOOLEAN columns/args one way or another. great in svn merge you can just reverse the order of change number to back out the patch, but there may be conflicts due to intervening changes. svn merge -r 1234:1233 https://svn.apache.org/repos/asf/db/derby/code/trunk >o If (1) and (2) are showstoppers, they are showstoppers regardless of what we do about user-declarable BOOLEAN columns/args. I don't think so. If we just leave it just like 10.1 is for now and think on this whole problem a bit more for how to create a BOOLEAN type for 10.3.
          Hide
          Rick Hillegas added a comment -

          Hi Kathey,

          I agree that this feature is not ready to be exposed to customers and I doubt that I will find time to finish it by our release date. I think that the server-to-client logic is pretty solid but I don't want to deal with compatiblity issues related to the unfinished client-to-server logic. Here I think the problem is that we are sending one-byte values from the server to the client but we are still sending 2 byte quantities in the other direction. Let me throw out some options for what we could do:

          1) We could back out the patch. I don't know how hard this is. If it's easy, it might be the best thing to do. Please advise me on this.

          2) We could disable the client-to-server compatibility problems by not allowing BOOLEAN as the type of columns and procedures/functions. Basically we could resurrect the disabling checks in the parser. For extra credit we could continue to test the new functionality under a tracepoint to make sure it does not regress as we move forward.

          I'm not sure I understand your issue with the metadata. Could you elaborate?

          Most of the illegal casts and comparisons are existing problems left over from Cloudscape. These illegal casts and comparisons occur in 10.1. The original effort to remove the BOOLEAN datatype was only partially successful. At the end of the day, we can forbid BOOLEAN columns but we can't remove the fundamental, resolving datatype of the WHERE clause. When considering bug 887, it is important to not balk at a gnat but swallow a camel. The newly introduced cases have caused us to stumble across something very old and very broken on which our production code relies. I don't think we can ignore bug 887 even if we back out 499. In my opinion, the presence or absence of the 499 code does not affect whether we should fix 887 for release 10.2.

          Let me punch up the importance of this point: Our own production code already relies on pre-existing illegal casts and comparisons. Quite likely, some of our customers do too.

          In summary:

          o I think that we should back out user-declarable BOOLEAN columns/args one way or another.

          o If (1) and (2) are showstoppers, they are showstoppers regardless of what we do about user-declarable BOOLEAN columns/args.

          Show
          Rick Hillegas added a comment - Hi Kathey, I agree that this feature is not ready to be exposed to customers and I doubt that I will find time to finish it by our release date. I think that the server-to-client logic is pretty solid but I don't want to deal with compatiblity issues related to the unfinished client-to-server logic. Here I think the problem is that we are sending one-byte values from the server to the client but we are still sending 2 byte quantities in the other direction. Let me throw out some options for what we could do: 1) We could back out the patch. I don't know how hard this is. If it's easy, it might be the best thing to do. Please advise me on this. 2) We could disable the client-to-server compatibility problems by not allowing BOOLEAN as the type of columns and procedures/functions. Basically we could resurrect the disabling checks in the parser. For extra credit we could continue to test the new functionality under a tracepoint to make sure it does not regress as we move forward. I'm not sure I understand your issue with the metadata. Could you elaborate? Most of the illegal casts and comparisons are existing problems left over from Cloudscape. These illegal casts and comparisons occur in 10.1. The original effort to remove the BOOLEAN datatype was only partially successful. At the end of the day, we can forbid BOOLEAN columns but we can't remove the fundamental, resolving datatype of the WHERE clause. When considering bug 887, it is important to not balk at a gnat but swallow a camel. The newly introduced cases have caused us to stumble across something very old and very broken on which our production code relies. I don't think we can ignore bug 887 even if we back out 499. In my opinion, the presence or absence of the 499 code does not affect whether we should fix 887 for release 10.2. Let me punch up the importance of this point: Our own production code already relies on pre-existing illegal casts and comparisons. Quite likely, some of our customers do too. In summary: o I think that we should back out user-declarable BOOLEAN columns/args one way or another. o If (1) and (2) are showstoppers, they are showstoppers regardless of what we do about user-declarable BOOLEAN columns/args.
          Hide
          Kathey Marsden added a comment -

          marking this issue to block DERBY-499.

          With Dan's review of the first checkin he noted that there was still work needed in the area of casting and comparisons for DERBY-499. Hopefully that work will resolve this issue.

          ignore previous link from 877. Slipped a digit

          Show
          Kathey Marsden added a comment - marking this issue to block DERBY-499 . With Dan's review of the first checkin he noted that there was still work needed in the area of casting and comparisons for DERBY-499 . Hopefully that work will resolve this issue. ignore previous link from 877. Slipped a digit
          Hide
          Kathey Marsden added a comment -

          Marking this issue to block DERBY-499.

          With Dan's review of the first checkin he noted that there was still work needed in the area of casting and comparisons for DERBY-499. Hopefully that work will resolve this issue.

          Show
          Kathey Marsden added a comment - Marking this issue to block DERBY-499 . With Dan's review of the first checkin he noted that there was still work needed in the area of casting and comparisons for DERBY-499 . Hopefully that work will resolve this issue.
          Hide
          Andrew McIntyre added a comment -

          I've committed the patch bug499_jdk13tests_rev2.diff with a couple of minor changes. I dropped the metadata_test.java diff, since it looked leftover from the previous patch and no longer needed. And instead of putting the jdk13 masters in a subdirectory, I put them at the top level.

          I'll take a look at the doc patch next.

          Show
          Andrew McIntyre added a comment - I've committed the patch bug499_jdk13tests_rev2.diff with a couple of minor changes. I dropped the metadata_test.java diff, since it looked leftover from the previous patch and no longer needed. And instead of putting the jdk13 masters in a subdirectory, I put them at the top level. I'll take a look at the doc patch next.
          Hide
          Rick Hillegas added a comment -

          I have attached a second rev of the patch to fix the problems in the jdk13 testrun: bug499_jdk13tests_rev2.diff. I hope that this patch addresses Kathey's concerns:

          1) Still tests the version related metadata calls.
          2) Allows the tests to pass on all jvms
          3) Keeps a single bumpversion target for release

          This version of the patch differs from the previous version as follows:

          o Re-instates the explicit printing of the version information from the metadata calls
          o Eliminates any need to change the odbc_metadata canons
          o Adds the new metadata canons to the list managed by the bumplastdigit release target in tools/release/build.xml

          I have cleanly run derbyall against this patch. I have also cleanly run the metadata and syscat tests in embedded, DerbyNetClient, and DerbyClient modes on jdk1.3 and jdk1.4. The patch contains the following:

          M tools\release\build.xml
          M java\testing\org\apache\derbyTesting\functionTests\tests\lang\syscat.sql
          M java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\metadata_test.java
          M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\syscat.out
          D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\metadata.out
          D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\syscat.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\metadata.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\syscat.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\metadata.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\syscat.out
          M java\testing\org\apache\derbyTesting\functionTests\master\syscat.out

          Show
          Rick Hillegas added a comment - I have attached a second rev of the patch to fix the problems in the jdk13 testrun: bug499_jdk13tests_rev2.diff. I hope that this patch addresses Kathey's concerns: 1) Still tests the version related metadata calls. 2) Allows the tests to pass on all jvms 3) Keeps a single bumpversion target for release This version of the patch differs from the previous version as follows: o Re-instates the explicit printing of the version information from the metadata calls o Eliminates any need to change the odbc_metadata canons o Adds the new metadata canons to the list managed by the bumplastdigit release target in tools/release/build.xml I have cleanly run derbyall against this patch. I have also cleanly run the metadata and syscat tests in embedded, DerbyNetClient, and DerbyClient modes on jdk1.3 and jdk1.4. The patch contains the following: M tools\release\build.xml M java\testing\org\apache\derbyTesting\functionTests\tests\lang\syscat.sql M java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\metadata_test.java M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\syscat.out D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\metadata.out D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\syscat.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13 A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\metadata.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\syscat.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\metadata.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\syscat.out M java\testing\org\apache\derbyTesting\functionTests\master\syscat.out
          Hide
          Rick Hillegas added a comment -

          It's, admittedly, an odd set of casts and I wonder if anyone really needs them. The casts between BOOLEAN and CLOB are inherited from the old Cloudscape days. It appears to me that these are required casts according to section 6.12 of volume 2 of the 2003 ANSI spec.

          Show
          Rick Hillegas added a comment - It's, admittedly, an odd set of casts and I wonder if anyone really needs them. The casts between BOOLEAN and CLOB are inherited from the old Cloudscape days. It appears to me that these are required casts according to section 6.12 of volume 2 of the 2003 ANSI spec.
          Hide
          Daniel John Debrunner added a comment -

          The functional spec lists BOOLEAN to and from CLOB as being a valid cast. This does not match the existing behaviour for CLOB where it can only be cast to/from a character type.

          Show
          Daniel John Debrunner added a comment - The functional spec lists BOOLEAN to and from CLOB as being a valid cast. This does not match the existing behaviour for CLOB where it can only be cast to/from a character type.
          Hide
          Rick Hillegas added a comment -

          Hi Kathey: The attached patch bug499_jdk13tests.diff fixes the regressions you found in the metadata.java and syscat.sql tests under jdk13:

          M java\testing\org\apache\derbyTesting\functionTests\tests\lang\syscat.sql
          M java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\metadata_test.java
          M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\metadata.out
          M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\syscat.out
          M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\odbc_metadata.out
          D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\metadata.out
          D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\syscat.out
          M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\odbc_metadata.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\metadata.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\syscat.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\metadata.out
          A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\syscat.out
          M java\testing\org\apache\derbyTesting\functionTests\master\metadata.out
          M java\testing\org\apache\derbyTesting\functionTests\master\syscat.out
          M java\testing\org\apache\derbyTesting\functionTests\master\odbc_metadata.out

          Along the way I made the following improvements to these tests:

          o I removed the printing of version numbers in the output of metadata.java. Instead, this test now compares version numbers internally. Keeping this test in sync with the release variables in dnc.properties now involves changing a couple constants in metadata.java rather than updating 10 canons. As extra credit, someone may want to further improve this test by removing the hard-coded version constants and, instead, get version information from Configuration.java. That depends on one's feelings about what the point of this version check is.

          o I removed a potential instability from syscat.sql by adding ORDER BY clauses to several queries whose results were not well-defined.

          My first attempt at fixing these regressions did not catch odbc_metadata. So I had 3 failures in derbyall: odbc_metadata under embedded, DerbyNetClient, and DerbyNet. I have fixed them. I have verified that all three tests (metadata, odbc_metadata, and syscat) run cleanly in all three configurations (embedded, DerbyNetClient, and DerbyNet) under jdk1.4 and jdk1.3.

          I believe there is a remaining piece of cleanup here: the j9_foundation canons for metadata and odbc_metadata. I would appreciate it if those canons could be fixed by someone who has installed the small device configuration.

          Show
          Rick Hillegas added a comment - Hi Kathey: The attached patch bug499_jdk13tests.diff fixes the regressions you found in the metadata.java and syscat.sql tests under jdk13: M java\testing\org\apache\derbyTesting\functionTests\tests\lang\syscat.sql M java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\metadata_test.java M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\metadata.out M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\syscat.out M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\odbc_metadata.out D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\metadata.out D java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\syscat.out M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\odbc_metadata.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13 A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\metadata.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk13\syscat.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\metadata.out A java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\jdk14\syscat.out M java\testing\org\apache\derbyTesting\functionTests\master\metadata.out M java\testing\org\apache\derbyTesting\functionTests\master\syscat.out M java\testing\org\apache\derbyTesting\functionTests\master\odbc_metadata.out Along the way I made the following improvements to these tests: o I removed the printing of version numbers in the output of metadata.java. Instead, this test now compares version numbers internally. Keeping this test in sync with the release variables in dnc.properties now involves changing a couple constants in metadata.java rather than updating 10 canons. As extra credit, someone may want to further improve this test by removing the hard-coded version constants and, instead, get version information from Configuration.java. That depends on one's feelings about what the point of this version check is. o I removed a potential instability from syscat.sql by adding ORDER BY clauses to several queries whose results were not well-defined. My first attempt at fixing these regressions did not catch odbc_metadata. So I had 3 failures in derbyall: odbc_metadata under embedded, DerbyNetClient, and DerbyNet. I have fixed them. I have verified that all three tests (metadata, odbc_metadata, and syscat) run cleanly in all three configurations (embedded, DerbyNetClient, and DerbyNet) under jdk1.4 and jdk1.3. I believe there is a remaining piece of cleanup here: the j9_foundation canons for metadata and odbc_metadata. I would appreciate it if those canons could be fixed by someone who has installed the small device configuration.
          Hide
          Kathey Marsden added a comment -

          For metadata.java, you may want to avoid additional canons because it would mean that you would also have to update the bumpversion target. (The output of this test changes with every new version). It would best I think if the difference can be handled programatically in the test.

          In terms of when this should be addressed, my personal opinion is that it is never good to have test failures for any significant length of time. When do you expect to submit your next patch?

          Show
          Kathey Marsden added a comment - For metadata.java, you may want to avoid additional canons because it would mean that you would also have to update the bumpversion target. (The output of this test changes with every new version). It would best I think if the difference can be handled programatically in the test. In terms of when this should be addressed, my personal opinion is that it is never good to have test failures for any significant length of time. When do you expect to submit your next patch?
          Hide
          Rick Hillegas added a comment -

          Hi Kathey: Thanks for finding this discrepancy. I think this is what's going on: I checked in new canons for these tests, which reflect the behavior of these tests with the Derby client under jdk 1.4. However, the old behavior of these tests is still preserved on jdk 1.3 (as documented in the functional spec for this feature). I believe that the appropriate action is to check in vm-specific canons for these tests to handle the 1.3 run. If you think this is urgent, I can pull together a patch for this. Otherwise, I can defer this cleanup to the next patch, which will address your other concerns.

          Show
          Rick Hillegas added a comment - Hi Kathey: Thanks for finding this discrepancy. I think this is what's going on: I checked in new canons for these tests, which reflect the behavior of these tests with the Derby client under jdk 1.4. However, the old behavior of these tests is still preserved on jdk 1.3 (as documented in the functional spec for this feature). I believe that the appropriate action is to check in vm-specific canons for these tests to handle the 1.3 run. If you think this is urgent, I can pull together a patch for this. Otherwise, I can defer this cleanup to the next patch, which will address your other concerns.
          Hide
          Kathey Marsden added a comment -

          Ignore previously deleted comment on this issue.

          There are two failures in jdk131 that I think might be related to the patch for this issue. Test output is attached as jdk131BooleanFailures.zip.

          derbyall/derbynetclientmats/derbynetmats.fail:jdbcapi/metadata.java
          derbyall/derbynetclientmats/derbynetmats.fail:lang/syscat.sql

          Show
          Kathey Marsden added a comment - Ignore previously deleted comment on this issue. There are two failures in jdk131 that I think might be related to the patch for this issue. Test output is attached as jdk131BooleanFailures.zip. derbyall/derbynetclientmats/derbynetmats.fail:jdbcapi/metadata.java derbyall/derbynetclientmats/derbynetmats.fail:lang/syscat.sql
          Hide
          Kathey Marsden added a comment -

          lang/syscat and jdbcapi/metadata test output for

          java version "1.3.1_15"
          Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1_15-b01)
          Java HotSpot(TM) Client VM (build 1.3.1_15-b01, mixed mode)

          Show
          Kathey Marsden added a comment - lang/syscat and jdbcapi/metadata test output for java version "1.3.1_15" Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1_15-b01) Java HotSpot(TM) Client VM (build 1.3.1_15-b01, mixed mode)
          Hide
          Daniel John Debrunner added a comment -

          Committed revision 358163 for patch bug499_rev4.diff.

          I think the BOOLEAN type should be added to some of the table driven tests like parameterMapping.java and casting.java before this jira entry is closed. In looking that the data type conversion routines the other day it seems
          that there are some strange conversions to/from BOOLEAN allowed. E.g. BOOLEAN to VARCHAR FOR BIT DATA.

          I did not commit the documentaton changes.

          Show
          Daniel John Debrunner added a comment - Committed revision 358163 for patch bug499_rev4.diff. I think the BOOLEAN type should be added to some of the table driven tests like parameterMapping.java and casting.java before this jira entry is closed. In looking that the data type conversion routines the other day it seems that there are some strange conversions to/from BOOLEAN allowed. E.g. BOOLEAN to VARCHAR FOR BIT DATA. I did not commit the documentaton changes.
          Hide
          Rick Hillegas added a comment -

          Thanks, Dan. I hope that this next patch addresses the class/filename and copyright problems: bug499_rev4.diff. I think that the previous rev corrected the class/filename in DRDAConstants.java. This version corrects the class/filenames in the remaining newly added source files: LangSuite.java and BooleanTest.java. I think that the copyright notices in those classes were ok to begin with. Let me know if anything else needs sanding down here.

          Show
          Rick Hillegas added a comment - Thanks, Dan. I hope that this next patch addresses the class/filename and copyright problems: bug499_rev4.diff. I think that the previous rev corrected the class/filename in DRDAConstants.java. This version corrects the class/filenames in the remaining newly added source files: LangSuite.java and BooleanTest.java. I think that the copyright notices in those classes were ok to begin with. Let me know if anything else needs sanding down here.
          Hide
          Daniel John Debrunner added a comment -

          You have the format correct, my last comment was just an example of one of the places you had the name wrong and it was an extract from svn diff. That's where the java/engine/org/apache/derby/iapi/reference/DRDAConstants.java (revision 0) comes from.

          Show
          Daniel John Debrunner added a comment - You have the format correct, my last comment was just an example of one of the places you had the name wrong and it was an extract from svn diff. That's where the java/engine/org/apache/derby/iapi/reference/DRDAConstants.java (revision 0) comes from.
          Hide
          Rick Hillegas added a comment -

          Hi Dan,

          It occurs to me that I may still have the class/filename wrong. Could you help me out? Here is the filename line from my latest attempt (bug499_rev3.diff):

          Derby - Class org.apache.derby.iapi.reference.DRDAConstants

          Unless I am losing my mind, this appears to follow the pattern for all the other classes in that package. For instance, here is the filename line from neighboring Attribute.java:

          Derby - Class org.apache.derby.iapi.reference.Attribute

          But this doesn't agree with the format in your last comment:

          java/engine/org/apache/derby/iapi/reference/DRDAConstants.java (revision 0)

          Can you point me at a file which models a correct pattern? Thanks.

          Show
          Rick Hillegas added a comment - Hi Dan, It occurs to me that I may still have the class/filename wrong. Could you help me out? Here is the filename line from my latest attempt (bug499_rev3.diff): Derby - Class org.apache.derby.iapi.reference.DRDAConstants Unless I am losing my mind, this appears to follow the pattern for all the other classes in that package. For instance, here is the filename line from neighboring Attribute.java: Derby - Class org.apache.derby.iapi.reference.Attribute But this doesn't agree with the format in your last comment: java/engine/org/apache/derby/iapi/reference/DRDAConstants.java (revision 0) Can you point me at a file which models a correct pattern? Thanks.
          Hide
          Rick Hillegas added a comment -

          Let's try this again. The bug499_rev3.diff patch addresses the filename issue.

          Show
          Rick Hillegas added a comment - Let's try this again. The bug499_rev3.diff patch addresses the filename issue.
          Hide
          Daniel John Debrunner added a comment -

          The file/class names in the copyright/licence header for the new files are still wrong:

          Eg.

          +++ java/engine/org/apache/derby/iapi/reference/DRDAConstants.java (revisio
          n 0)
          @@ -0,0 +1,215 @@
          +/*
          +
          + Derby - Class org.apache.derbyTesting.functionTests.tests.compatibility.JDBC
          DriverTest
          +

          Show
          Daniel John Debrunner added a comment - The file/class names in the copyright/licence header for the new files are still wrong: Eg. +++ java/engine/org/apache/derby/iapi/reference/DRDAConstants.java (revisio n 0) @@ -0,0 +1,215 @@ +/* + + Derby - Class org.apache.derbyTesting.functionTests.tests.compatibility.JDBC DriverTest +
          Hide
          Rick Hillegas added a comment -

          Fix the copyright notice as Dan requested. The new patch is called bug499_rev2.diff. If you diff it against the previous patch, you will see that this is the only difference.

          Show
          Rick Hillegas added a comment - Fix the copyright notice as Dan requested. The new patch is called bug499_rev2.diff. If you diff it against the previous patch, you will see that this is the only difference.
          Hide
          Daniel John Debrunner added a comment -

          Patch passes the tests but has some minor issues:

          The file name in the copyright/licence header for the new files does not match the file name.

          The new DRDA constants file has Copyright Rick Hillegas which is not acceptable to the ASF.

          Show
          Daniel John Debrunner added a comment - Patch passes the tests but has some minor issues: The file name in the copyright/licence header for the new files does not match the file name. The new DRDA constants file has Copyright Rick Hillegas which is not acceptable to the ASF.
          Hide
          Rick Hillegas added a comment -

          Various changes have accumulated in the mainline since I posted this patch a week ago. I have merged in those changes and run derbyall cleanly. I am reattaching the merged patch.

          Show
          Rick Hillegas added a comment - Various changes have accumulated in the mainline since I posted this patch a week ago. I have merged in those changes and run derbyall cleanly. I am reattaching the merged patch.
          Hide
          Rick Hillegas added a comment -

          I have made the parser change which Satheesh pointed out. I also removed new DRDA datatypes not actually needed by this submission as Kathey requested. To accomodate the parser change, I had to update a couple unit tests:

          M java\engine\org\apache\derby\impl\sql\compile\sqlgrammar.jj
          A java\engine\org\apache\derby\iapi\reference\DRDAConstants.java
          M java\drda\org\apache\derby\impl\drda\FdocaConstants.java
          M java\drda\org\apache\derby\impl\drda\SQLTypes.java
          M java\drda\org\apache\derby\impl\drda\EXTDTAInputStream.java
          M java\drda\org\apache\derby\impl\drda\NetworkServerControlImpl.java
          M java\drda\org\apache\derby\impl\drda\AppRequester.java
          M java\drda\org\apache\derby\impl\drda\DRDAConnThread.java
          M java\testing\org\apache\derbyTesting\functionTests\tests\lang\aggregate.sql
          M java\testing\org\apache\derbyTesting\functionTests\tests\lang\subquery.sql
          M java\testing\org\apache\derbyTesting\functionTests\tests\lang\db2Compatibility.sql
          M java\testing\org\apache\derbyTesting\functionTests\tests\lang\logop.sql
          M java\testing\org\apache\derbyTesting\functionTests\tests\lang\schemas.sql
          A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang
          A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\BooleanTest.java
          A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\default_app.properties
          A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\LangSuite.java
          A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\build.xml
          M java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\CompatibilitySuite.java
          M java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\testScript.xml
          M java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\JDBCDriverTest.java
          M java\testing\org\apache\derbyTesting\functionTests\master\subquery.out
          M java\testing\org\apache\derbyTesting\functionTests\master\db2Compatibility.out
          M java\testing\org\apache\derbyTesting\functionTests\master\implicitConversions.out
          M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\metadata.out
          M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\syscat.out
          M java\testing\org\apache\derbyTesting\functionTests\master\logop.out
          M java\testing\org\apache\derbyTesting\functionTests\master\valuesclause.out
          M java\testing\org\apache\derbyTesting\functionTests\master\schemas.out
          M java\testing\org\apache\derbyTesting\functionTests\master\aggregate.out
          M java\testing\org\apache\derbyTesting\functionTests\suites\derbylang.runall
          M java\testing\org\apache\derbyTesting\functionTests\util\DerbyJUnitTest.java
          M java\testing\build.xml
          M java\client\org\apache\derby\client\net\FdocaConstants.java
          M java\client\org\apache\derby\client\net\NetConfiguration.java
          M java\client\org\apache\derby\client\net\Typdef.java
          M java\client\org\apache\derby\client\net\NetStatementRequest.java
          M java\client\org\apache\derby\client\am\CrossConverters.java
          M java\client\org\apache\derby\client\am\Types.java
          M java\client\org\apache\derby\client\am\Cursor.java
          M java\client\org\apache\derby\client\am\SignedBinary.java
          M java\client\org\apache\derby\client\am\DatabaseMetaData.java
          M java\client\org\apache\derby\client\am\ColumnMetaData.java

          Once this first increment is committed, I will work on another patch to address Kathey's other concerns.

          Show
          Rick Hillegas added a comment - I have made the parser change which Satheesh pointed out. I also removed new DRDA datatypes not actually needed by this submission as Kathey requested. To accomodate the parser change, I had to update a couple unit tests: M java\engine\org\apache\derby\impl\sql\compile\sqlgrammar.jj A java\engine\org\apache\derby\iapi\reference\DRDAConstants.java M java\drda\org\apache\derby\impl\drda\FdocaConstants.java M java\drda\org\apache\derby\impl\drda\SQLTypes.java M java\drda\org\apache\derby\impl\drda\EXTDTAInputStream.java M java\drda\org\apache\derby\impl\drda\NetworkServerControlImpl.java M java\drda\org\apache\derby\impl\drda\AppRequester.java M java\drda\org\apache\derby\impl\drda\DRDAConnThread.java M java\testing\org\apache\derbyTesting\functionTests\tests\lang\aggregate.sql M java\testing\org\apache\derbyTesting\functionTests\tests\lang\subquery.sql M java\testing\org\apache\derbyTesting\functionTests\tests\lang\db2Compatibility.sql M java\testing\org\apache\derbyTesting\functionTests\tests\lang\logop.sql M java\testing\org\apache\derbyTesting\functionTests\tests\lang\schemas.sql A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\BooleanTest.java A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\default_app.properties A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\LangSuite.java A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\lang\build.xml M java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\CompatibilitySuite.java M java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\testScript.xml M java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\JDBCDriverTest.java M java\testing\org\apache\derbyTesting\functionTests\master\subquery.out M java\testing\org\apache\derbyTesting\functionTests\master\db2Compatibility.out M java\testing\org\apache\derbyTesting\functionTests\master\implicitConversions.out M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\metadata.out M java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\syscat.out M java\testing\org\apache\derbyTesting\functionTests\master\logop.out M java\testing\org\apache\derbyTesting\functionTests\master\valuesclause.out M java\testing\org\apache\derbyTesting\functionTests\master\schemas.out M java\testing\org\apache\derbyTesting\functionTests\master\aggregate.out M java\testing\org\apache\derbyTesting\functionTests\suites\derbylang.runall M java\testing\org\apache\derbyTesting\functionTests\util\DerbyJUnitTest.java M java\testing\build.xml M java\client\org\apache\derby\client\net\FdocaConstants.java M java\client\org\apache\derby\client\net\NetConfiguration.java M java\client\org\apache\derby\client\net\Typdef.java M java\client\org\apache\derby\client\net\NetStatementRequest.java M java\client\org\apache\derby\client\am\CrossConverters.java M java\client\org\apache\derby\client\am\Types.java M java\client\org\apache\derby\client\am\Cursor.java M java\client\org\apache\derby\client\am\SignedBinary.java M java\client\org\apache\derby\client\am\DatabaseMetaData.java M java\client\org\apache\derby\client\am\ColumnMetaData.java Once this first increment is committed, I will work on another patch to address Kathey's other concerns.
          Hide
          Rick Hillegas added a comment -

          Satheesh, thank you for pointing out the disabling code in the inSelectClause() production. I will fix this and add appropriate test cases.

          Kathey, thank you for going through the network code so carefully. Some responses follow:

          0) I understand your misgivings. I do however see some value in rolling some ongoing cleanup into patches. I tried to steer a middle course here: I moved the datatype ids to a common file but I deferred the rototill of Typdef.java to a later submission. I hope this is ok. Please let me know if you think this is a showstopper.

          1) I certainly hope that the Open Group will finalize the new datatypes in the 10.2 timeframe. It's a slow process. I don't see a conflict with Army's work on XML but Army and I can work together on this.

          You are right, that if we include the DRDA types, we should include corresponding DB2 types. I will add these as more placeholders.

          2) Yes, the BOOLEANS are coming from the client as SMALLINT. I agree that it would be better to send these as booleans. I will make that change. I will also make the other changes you suggest here.

          3) I can add the CrossConverter code you suggest. But I can't answer your question: Obviously the tests I have written don't stress this code. Can you help me understand what test cases will stress this code?

          4) Thanks for catching this. I will add BOOLEAN to getTypeInfo().

          5i) I ran the compatibility suite in the following combinations:

          o I allowed the server to range over the following options: 10.0.2.1, 10.1.1.0, 10.1.2.0, and mainline.
          o Each server was run on the following vms: 1.3, 1.4, and 1.5
          o For each combination of server and server vm, I allowed the client to range over the following options: DB2JCC, 10.1.1.0, 10.1.2.0, and mainline.
          o For each combination of server, server vm, and client, I ran the client on the following vms: 1.3, 1.4, and 1.5.

          In addition, I ran the mainline in an embedded configuration on the following vms: 1.3, 1.4, and 1.5.

          5ii) Thanks for suggesting this. I will add some boolean cases to an existing ij test to verify formatting. I have already tested with older clients and verified that BOOLEAN goes to old clients as SMALLINT.

          6) Thanks for this suggestion too. I will add boolean cases to jdbcapi/parameterMapping.

          7) I welcome your creative suggestions here. The best idea I have been able to come up with is the running of the compatibility suite as part of some nightly or weekly verification.

          Show
          Rick Hillegas added a comment - Satheesh, thank you for pointing out the disabling code in the inSelectClause() production. I will fix this and add appropriate test cases. Kathey, thank you for going through the network code so carefully. Some responses follow: 0) I understand your misgivings. I do however see some value in rolling some ongoing cleanup into patches. I tried to steer a middle course here: I moved the datatype ids to a common file but I deferred the rototill of Typdef.java to a later submission. I hope this is ok. Please let me know if you think this is a showstopper. 1) I certainly hope that the Open Group will finalize the new datatypes in the 10.2 timeframe. It's a slow process. I don't see a conflict with Army's work on XML but Army and I can work together on this. You are right, that if we include the DRDA types, we should include corresponding DB2 types. I will add these as more placeholders. 2) Yes, the BOOLEANS are coming from the client as SMALLINT. I agree that it would be better to send these as booleans. I will make that change. I will also make the other changes you suggest here. 3) I can add the CrossConverter code you suggest. But I can't answer your question: Obviously the tests I have written don't stress this code. Can you help me understand what test cases will stress this code? 4) Thanks for catching this. I will add BOOLEAN to getTypeInfo(). 5i) I ran the compatibility suite in the following combinations: o I allowed the server to range over the following options: 10.0.2.1, 10.1.1.0, 10.1.2.0, and mainline. o Each server was run on the following vms: 1.3, 1.4, and 1.5 o For each combination of server and server vm, I allowed the client to range over the following options: DB2JCC, 10.1.1.0, 10.1.2.0, and mainline. o For each combination of server, server vm, and client, I ran the client on the following vms: 1.3, 1.4, and 1.5. In addition, I ran the mainline in an embedded configuration on the following vms: 1.3, 1.4, and 1.5. 5ii) Thanks for suggesting this. I will add some boolean cases to an existing ij test to verify formatting. I have already tested with older clients and verified that BOOLEAN goes to old clients as SMALLINT. 6) Thanks for this suggestion too. I will add boolean cases to jdbcapi/parameterMapping. 7) I welcome your creative suggestions here. The best idea I have been able to come up with is the running of the compatibility suite as part of some nightly or weekly verification.
          Hide
          Kathey Marsden added a comment -

          Looks like good progress on Boolean.

          Code Comments and questions

          0)
          I do kind of wish constant reorganization and partial addition of other new types were in separate patches.

          1)
          My biggest question about the patch are the new types and the assignment of these temporary codes. Will these be finalized before 10.2? Why have some other types be added to the DRDATypes but not the SQL Types. I was wondering if it might pose any conflict with work underway for XML?

          // Experimental types. These codes will change when the Open Group
          // publishes an addendum to the DRDA spec covering these
          // datatypes.

          public static final int DRDA_TYPE_BOOLEAN = 0xD0;
          public static final int DRDA_TYPE_NBOOLEAN = 0xD1;
          public static final int DRDA_TYPE_XML = 0xD2;
          public static final int DRDA_TYPE_NXML = 0xD3;
          public static final int DRDA_TYPE_NATCHAR = 0xD4;
          public static final int DRDA_TYPE_NNATCHAR = 0xD5;
          public static final int DRDA_TYPE_NATVARCHAR = 0xD6;
          public static final int DRDA_TYPE_NNATVARCHAR = 0xD7;
          public static final int DRDA_TYPE_LONGNATVARCHAR = 0xD8;
          public static final int DRDA_TYPE_NLONGNATVARCHAR = 0xD9;
          public static final int DRDA_TYPE_NATCLOB = 0xDA;
          public static final int DRDA_TYPE_NNATCLOB = 0xDB;

          // extensions to the db2 datatypes
          public static final int DB2_SQLTYPE_BOOLEAN = 1000; // BOOLEAN
          public static final int DB2_SQLTYPE_NBOOLEAN = 1001;

          }

          2)
          DRDAConnThread comments

          In readAndSetParams it looks like BOOLEAN is not being handled. Are the client parameters still being sent as SMALLINT parameters?

          Line 6576
          I think rather than a makeint method in DRDAConnThread I would rather see a writeByte(boolean b) method in DDMWriter similar to writeShort(boolean b),

          changing
          writer.writeByte( makeInt( (Boolean) val ) );

          to
          writer.writeByte((Boolean) val ).booleanValue() );

          3)
          There seem to be places in the client code that have case statements for all the types, for example, many of the methods in CrossConverters. Does boolean need to be handled in these cases?

          4)
          DatabaseMetadata.getTypeInfo() doesn't seem to return BOOLEAN/BIT. I think there has to be a differentiation of the output based on the client JVM version.

          Test comments

          5)
          What client/server version combinations did you run the Compatibility tests?

          5)
          Would you consider adding a boolean column to an ij test in the netmats suite like supersimple.sql to verify the ij formatting is ok and also provide some testing with earlier clients that it is being sent as smallint for clients other than derby 10.2 client?

          6)
          Should jdbcapi/parameterMapping.java be updated for the new type? I know it is not junit but does provide a comprehensive picture of the type behaviour and it shouldn't be too hard to add a type to that test.

          7)
          It seems more important than ever that we have at least some very basic client/server compatibility testing with the the 10.1.1.0 release as part of derbyall.

          Show
          Kathey Marsden added a comment - Looks like good progress on Boolean. Code Comments and questions 0) I do kind of wish constant reorganization and partial addition of other new types were in separate patches. 1) My biggest question about the patch are the new types and the assignment of these temporary codes. Will these be finalized before 10.2? Why have some other types be added to the DRDATypes but not the SQL Types. I was wondering if it might pose any conflict with work underway for XML? // Experimental types. These codes will change when the Open Group // publishes an addendum to the DRDA spec covering these // datatypes. public static final int DRDA_TYPE_BOOLEAN = 0xD0; public static final int DRDA_TYPE_NBOOLEAN = 0xD1; public static final int DRDA_TYPE_XML = 0xD2; public static final int DRDA_TYPE_NXML = 0xD3; public static final int DRDA_TYPE_NATCHAR = 0xD4; public static final int DRDA_TYPE_NNATCHAR = 0xD5; public static final int DRDA_TYPE_NATVARCHAR = 0xD6; public static final int DRDA_TYPE_NNATVARCHAR = 0xD7; public static final int DRDA_TYPE_LONGNATVARCHAR = 0xD8; public static final int DRDA_TYPE_NLONGNATVARCHAR = 0xD9; public static final int DRDA_TYPE_NATCLOB = 0xDA; public static final int DRDA_TYPE_NNATCLOB = 0xDB; // extensions to the db2 datatypes public static final int DB2_SQLTYPE_BOOLEAN = 1000; // BOOLEAN public static final int DB2_SQLTYPE_NBOOLEAN = 1001; } 2) DRDAConnThread comments In readAndSetParams it looks like BOOLEAN is not being handled. Are the client parameters still being sent as SMALLINT parameters? Line 6576 I think rather than a makeint method in DRDAConnThread I would rather see a writeByte(boolean b) method in DDMWriter similar to writeShort(boolean b), changing writer.writeByte( makeInt( (Boolean) val ) ); to writer.writeByte((Boolean) val ).booleanValue() ); 3) There seem to be places in the client code that have case statements for all the types, for example, many of the methods in CrossConverters. Does boolean need to be handled in these cases? 4) DatabaseMetadata.getTypeInfo() doesn't seem to return BOOLEAN/BIT. I think there has to be a differentiation of the output based on the client JVM version. Test comments 5) What client/server version combinations did you run the Compatibility tests? 5) Would you consider adding a boolean column to an ij test in the netmats suite like supersimple.sql to verify the ij formatting is ok and also provide some testing with earlier clients that it is being sent as smallint for clients other than derby 10.2 client? 6) Should jdbcapi/parameterMapping.java be updated for the new type? I know it is not junit but does provide a comprehensive picture of the type behaviour and it shouldn't be too hard to add a type to that test. 7) It seems more important than ever that we have at least some very basic client/server compatibility testing with the the 10.1.1.0 release as part of derbyall.
          Hide
          Satheesh Bandaram added a comment -

          This patch doesn't allow all boolean expressions. Following would fail:

          Select a>b from t
          values myFunc(current date > '1/1/99')

          If you look at additiveExpression() function in sqlgrammar.jj, it has a third parameter 'inSelectClause'. This is currently used to prevent sending boolean values to clients, since there is no support for Boolean. (before the patch)

          Show
          Satheesh Bandaram added a comment - This patch doesn't allow all boolean expressions. Following would fail: Select a>b from t values myFunc(current date > '1/1/99') If you look at additiveExpression() function in sqlgrammar.jj, it has a third parameter 'inSelectClause'. This is currently used to prevent sending boolean values to clients, since there is no support for Boolean. (before the patch)
          Hide
          Rick Hillegas added a comment -

          Attached new rev of patch (bug499.diff) addressing initial issues with BooleanTest raised by Dan: 1) BooleanTest now extends DerbyJUnitTest rather than LangSuite. 2) Used Integers rather than Shorts to model SMALLINTs in the assertRow() calls which track casts from BOOLEAN to numeric and string types. New rev of BooleanTest runs cleanly.

          Show
          Rick Hillegas added a comment - Attached new rev of patch (bug499.diff) addressing initial issues with BooleanTest raised by Dan: 1) BooleanTest now extends DerbyJUnitTest rather than LangSuite. 2) Used Integers rather than Shorts to model SMALLINTs in the assertRow() calls which track casts from BOOLEAN to numeric and string types. New rev of BooleanTest runs cleanly.
          Hide
          Rick Hillegas added a comment -

          I am attaching bug499_doc.zip, which contains changes to the Tuning and Reference guides. This zip file contains:

          o bug499_doc.diff - the svn diffs for these doc changes
          o ref - a directory containing the html for the Reference guide, including these changes

          I successfully built the html and pdf versions of these two documents. However, I had to increase the vm's memory in order to build the pdf version of the Reference guide. I didn't explore how much to increase the memory. I simply set the ant options as follows before invoking the build:

          export ANT_OPTS="-Xmx512m"

          Here's a summary of the doc changes I made:

          TUNING GUIDE

          I added boolean to the table of getXXX() and setXXX() methods described in the section titled "Use the appropriate getXXX and setXXX methods for the type"

          REFERENCE GUIDE

          I added a section on boolean to the "Data type" roundup.

          I also added a row and column for boolean to the "CAST" page under "Built-in functions". I also added to that page a paragraph describing the legal conversions to and from boolean.

          Show
          Rick Hillegas added a comment - I am attaching bug499_doc.zip, which contains changes to the Tuning and Reference guides. This zip file contains: o bug499_doc.diff - the svn diffs for these doc changes o ref - a directory containing the html for the Reference guide, including these changes I successfully built the html and pdf versions of these two documents. However, I had to increase the vm's memory in order to build the pdf version of the Reference guide. I didn't explore how much to increase the memory. I simply set the ant options as follows before invoking the build: export ANT_OPTS="-Xmx512m" Here's a summary of the doc changes I made: TUNING GUIDE I added boolean to the table of getXXX() and setXXX() methods described in the section titled "Use the appropriate getXXX and setXXX methods for the type" REFERENCE GUIDE I added a section on boolean to the "Data type" roundup. I also added a row and column for boolean to the "CAST" page under "Built-in functions". I also added to that page a paragraph describing the legal conversions to and from boolean.
          Hide
          Rick Hillegas added a comment -

          This patch (bug499.diff) enables the BOOLEAN datatype as described in the functional
          spec (BooleanFS.html). On my machine, this patch passes the derbyall
          and compatibility suite tests. This patch affects the following files:

          M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj
          A java/engine/org/apache/derby/iapi/reference/DRDAConstants.java
          M java/drda/org/apache/derby/impl/drda/FdocaConstants.java
          M java/drda/org/apache/derby/impl/drda/SQLTypes.java
          M java/drda/org/apache/derby/impl/drda/EXTDTAInputStream.java
          M java/drda/org/apache/derby/impl/drda/NetworkServerControlImpl.java
          M java/drda/org/apache/derby/impl/drda/DRDAConnThread.java
          M java/drda/org/apache/derby/impl/drda/AppRequester.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/db2Compatibility.sql
          A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang
          A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/BooleanTest.java
          A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/default_app.properties
          A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/LangSuite.java
          A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/build.xml
          M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/CompatibilitySuite.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/testScript.xml
          M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/JDBCDriverTest.java
          M java/testing/org/apache/derbyTesting/functionTests/master/db2Compatibility.out
          M java/testing/org/apache/derbyTesting/functionTests/master/implicitConversions.out
          M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/metadata.out
          M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/syscat.out
          M java/testing/org/apache/derbyTesting/functionTests/suites/derbylang.runall
          M java/testing/org/apache/derbyTesting/functionTests/util/DerbyJUnitTest.java
          M java/testing/build.xml
          M java/client/org/apache/derby/client/net/FdocaConstants.java
          M java/client/org/apache/derby/client/net/NetConfiguration.java
          M java/client/org/apache/derby/client/net/Typdef.java
          M java/client/org/apache/derby/client/net/NetStatementRequest.java
          M java/client/org/apache/derby/client/am/CrossConverters.java
          M java/client/org/apache/derby/client/am/Types.java
          M java/client/org/apache/derby/client/am/Cursor.java
          M java/client/org/apache/derby/client/am/SignedBinary.java
          M java/client/org/apache/derby/client/am/DatabaseMetaData.java
          M java/client/org/apache/derby/client/am/ColumnMetaData.java

          A brief description of these changes follows:

          1) sqlgrammar.jj - Small changes here to enable the BOOLEAN type and
          its literals: true, false, and unknown.

          2) DRDAConstants.java - This new file holds common definitions of the
          DRDA and DB2 datatypes plus Derby's product identifiers. Client and
          server code now share these constants. This file adds new DRDA
          datatypes for BOOLEAN as well as for other datatypes we may add in
          10.2. The Open Group is almost at the point where I can submit
          these types for approval.

          Note that an additional piece of cleanup can be done later: Right
          now, Typdef.java has a long table (environmentTables_) of magic
          numbers which are really just the DRDA datatypes. This table should
          be scrubbed to use the constants from DRDAConstants.java.

          3) AppRequester.java - This server class, which already held product version
          info, now reports whether the connected client can handle the new
          BOOLEAN datatype.

          4) Various other changes to the DRDA network layer to recognize
          BOOLEAN and handle it correctly based on the client's capabilities.

          5) JDBCDriverTest.java - Additional compatibility tests added to track
          assertions from the functional spec.

          6) LangSuite.java - A new JUnit suite currently containing one test:
          BooleanTest.java. LangSuite itself is wired into the derbylang
          suite. I'm thinking that over time, we can migrate existing derbylang
          tests into this suite and also wire in new JUnit-based language
          tests.

          7) BooleanTest.java - SQL tests for the BOOLEAN datatype. These
          supplement the JDBC tests for BOOLEAN found in JDBCDriverTest.

          8) testScript.xml - Now tracks the trunk/embedded combination on
          jdk1.3 and 1.5.

          I hope that when this submission is committed, it will be complete
          enough that we can use it as a template for adding additional
          datatypes (such as XML) to our network layer.

          A later submission will address changes to the user docs.

          Show
          Rick Hillegas added a comment - This patch (bug499.diff) enables the BOOLEAN datatype as described in the functional spec (BooleanFS.html). On my machine, this patch passes the derbyall and compatibility suite tests. This patch affects the following files: M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj A java/engine/org/apache/derby/iapi/reference/DRDAConstants.java M java/drda/org/apache/derby/impl/drda/FdocaConstants.java M java/drda/org/apache/derby/impl/drda/SQLTypes.java M java/drda/org/apache/derby/impl/drda/EXTDTAInputStream.java M java/drda/org/apache/derby/impl/drda/NetworkServerControlImpl.java M java/drda/org/apache/derby/impl/drda/DRDAConnThread.java M java/drda/org/apache/derby/impl/drda/AppRequester.java M java/testing/org/apache/derbyTesting/functionTests/tests/lang/db2Compatibility.sql A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/BooleanTest.java A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/default_app.properties A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/LangSuite.java A java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/lang/build.xml M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/CompatibilitySuite.java M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/testScript.xml M java/testing/org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/JDBCDriverTest.java M java/testing/org/apache/derbyTesting/functionTests/master/db2Compatibility.out M java/testing/org/apache/derbyTesting/functionTests/master/implicitConversions.out M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/metadata.out M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/syscat.out M java/testing/org/apache/derbyTesting/functionTests/suites/derbylang.runall M java/testing/org/apache/derbyTesting/functionTests/util/DerbyJUnitTest.java M java/testing/build.xml M java/client/org/apache/derby/client/net/FdocaConstants.java M java/client/org/apache/derby/client/net/NetConfiguration.java M java/client/org/apache/derby/client/net/Typdef.java M java/client/org/apache/derby/client/net/NetStatementRequest.java M java/client/org/apache/derby/client/am/CrossConverters.java M java/client/org/apache/derby/client/am/Types.java M java/client/org/apache/derby/client/am/Cursor.java M java/client/org/apache/derby/client/am/SignedBinary.java M java/client/org/apache/derby/client/am/DatabaseMetaData.java M java/client/org/apache/derby/client/am/ColumnMetaData.java A brief description of these changes follows: 1) sqlgrammar.jj - Small changes here to enable the BOOLEAN type and its literals: true, false, and unknown. 2) DRDAConstants.java - This new file holds common definitions of the DRDA and DB2 datatypes plus Derby's product identifiers. Client and server code now share these constants. This file adds new DRDA datatypes for BOOLEAN as well as for other datatypes we may add in 10.2. The Open Group is almost at the point where I can submit these types for approval. Note that an additional piece of cleanup can be done later: Right now, Typdef.java has a long table (environmentTables_) of magic numbers which are really just the DRDA datatypes. This table should be scrubbed to use the constants from DRDAConstants.java. 3) AppRequester.java - This server class, which already held product version info, now reports whether the connected client can handle the new BOOLEAN datatype. 4) Various other changes to the DRDA network layer to recognize BOOLEAN and handle it correctly based on the client's capabilities. 5) JDBCDriverTest.java - Additional compatibility tests added to track assertions from the functional spec. 6) LangSuite.java - A new JUnit suite currently containing one test: BooleanTest.java. LangSuite itself is wired into the derbylang suite. I'm thinking that over time, we can migrate existing derbylang tests into this suite and also wire in new JUnit-based language tests. 7) BooleanTest.java - SQL tests for the BOOLEAN datatype. These supplement the JDBC tests for BOOLEAN found in JDBCDriverTest. 8) testScript.xml - Now tracks the trunk/embedded combination on jdk1.3 and 1.5. I hope that when this submission is committed, it will be complete enough that we can use it as a template for adding additional datatypes (such as XML) to our network layer. A later submission will address changes to the user docs.
          Hide
          Rick Hillegas added a comment -

          Update functional spec: Responses to various comments. Added UNKNOWN literal. Added section on casts. JDBC API determined by matrix of conditions. Described results of getXXX() calls and limitations.

          Show
          Rick Hillegas added a comment - Update functional spec: Responses to various comments. Added UNKNOWN literal. Added section on casts. JDBC API determined by matrix of conditions. Described results of getXXX() calls and limitations.
          Hide
          Kathey Marsden added a comment -

          It would be good to document the value that BOOLEAN true/false will return when returned as SMALLINT to pre 10.2 drda clients, and non-Derby clients. Clarification of getXXX, setXXX and metadata would be good too and is being discussed on this thread.
          http://www.nabble.com/Re%3A-jira-Updated%3A-%28DERBY-499%29-Expose-BOOLEAN-datatype-to-end-users-t593782.html

          Show
          Kathey Marsden added a comment - It would be good to document the value that BOOLEAN true/false will return when returned as SMALLINT to pre 10.2 drda clients, and non-Derby clients. Clarification of getXXX, setXXX and metadata would be good too and is being discussed on this thread. http://www.nabble.com/Re%3A-jira-Updated%3A-%28DERBY-499%29-Expose-BOOLEAN-datatype-to-end-users-t593782.html
          Hide
          Rick Hillegas added a comment -

          I have attached a thin functional specification describing expected SQL and JDBC behavior for the re-enabled BOOLEAN datatype.

          Show
          Rick Hillegas added a comment - I have attached a thin functional specification describing expected SQL and JDBC behavior for the re-enabled BOOLEAN datatype.
          Hide
          Rick Hillegas added a comment -

          Hi Myrna,

          Do you know of any specific support that's missing or are you just concerned that something might not work? Any specifics you have here would help. Thanks.

          Show
          Rick Hillegas added a comment - Hi Myrna, Do you know of any specific support that's missing or are you just concerned that something might not work? Any specifics you have here would help. Thanks.

            People

            • Assignee:
              Unassigned
              Reporter:
              Rick Hillegas
            • Votes:
              16 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development