Derby
  1. Derby
  2. DERBY-3181

isNullable on ResultSetMetaData from DatabaseMetaData.getBestRowIdentifier values are opposite when there is no rows in ResultSet vs. when there is a row.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 10.4.1.3
    • Fix Version/s: 10.13.0.0
    • Component/s: JDBC
    • Urgency:
      Normal
    • Issue & fix info:
      Newcomer, Repro attached

      Description

      With code like the following:

      DatabaseMetaData dmd = conn.getMetaData();
      ResultSet rs = dmd.getBestRowIdentifier(null,"APP","a",3,true);
      ResultSetMetaData rsmd = rs.getMetaData();
      int actualCols = rsmd.getColumnCount();
      for (int i = 0; i < actualCols; i++)

      { System.out.print("getColumnName: " + rsmd.getColumnName(i+1) + ", isNullable: "); System.out.println(rsmd.isNullable(i+1)); }

      The printed values for isNullable returned are opposite of what they are when the getBestRowIdentifier call looks like this:
      ResultSet rs = dmd.getBestRowIdentifier(null,"APP","a",1,true);

      In the latter case, the values are:
      getColumnName: SCOPE, isNullable: 0
      getColumnName: COLUMN_NAME, isNullable: 1
      getColumnName: DATA_TYPE, isNullable: 0
      getColumnName: TYPE_NAME, isNullable: 1
      getColumnName: COLUMN_SIZE, isNullable: 0
      getColumnName: BUFFER_LENGTH, isNullable: 0
      getColumnName: DECIMAL_DIGITS, isNullable: 0
      getColumnName: PSEUDO_COLUMN, isNullable: 0

      In the first case, the values are:
      getColumnName: SCOPE, isNullable: 1
      getColumnName: COLUMN_NAME, isNullable: 0
      getColumnName: DATA_TYPE, isNullable: 1
      getColumnName: TYPE_NAME, isNullable: 1
      getColumnName: COLUMN_SIZE, isNullable: 1
      getColumnName: BUFFER_LENGTH, isNullable: 1
      getColumnName: DECIMAL_DIGITS, isNullable: 1
      getColumnName: PSEUDO_COLUMN, isNullable: 1

      The isNullable value should be stable.
      It's probably worthwhile verifying what the value should be in the first place.

      1. deleteQuery.diff
        1 kB
        Danoja Dias
      2. Derby3181.diff
        3 kB
        Danoja Dias
      3. testChange.diff
        1 kB
        Bryan Pendleton
      4. Derby-3181.diff
        0.6 kB
        Danoja Dias
      5. repro.java
        5 kB
        Myrna van Lunteren

        Activity

        Hide
        Bryan Pendleton added a comment -

        We've completed the intended work for this issue; resolving.

        Show
        Bryan Pendleton added a comment - We've completed the intended work for this issue; resolving.
        Hide
        ASF subversion and git services added a comment -

        Commit 1746487 from Bryan Pendleton in branch 'code/trunk'
        [ https://svn.apache.org/r1746487 ]

        DERBY-3181: DatabaseMetaData.getBestRowIdentifier behavior with invalid scope

        This patch was contributed by Danoja Dias (danojadias at gmail dot com)

        This change is a follow-on to revision 1745414, and removes the
        getBestRowIdentifierEmpty query definition from the queries in
        metadata.properties.

        Show
        ASF subversion and git services added a comment - Commit 1746487 from Bryan Pendleton in branch 'code/trunk' [ https://svn.apache.org/r1746487 ] DERBY-3181 : DatabaseMetaData.getBestRowIdentifier behavior with invalid scope This patch was contributed by Danoja Dias (danojadias at gmail dot com) This change is a follow-on to revision 1745414, and removes the getBestRowIdentifierEmpty query definition from the queries in metadata.properties.
        Hide
        Danoja Dias added a comment -

        metadata.properties file is changed and all tests passed.
        I attached the diff file.

        Show
        Danoja Dias added a comment - metadata.properties file is changed and all tests passed. I attached the diff file.
        Hide
        Bryan Pendleton added a comment -

        I think that should be all we need to do, but I am not certain.

        Can you please try removing that section as an experiment, and maybe
        do a clean build and some tests, to see if you encounter any problems?

        In the meantime, I think that your existing patch is a clear
        improvement over the current behavior, so I have committed it.

        I suggest that we leave this issue open while you investigate
        whether the query definition can be removed from the properties
        file, or whether it is not as simple as that.

        Show
        Bryan Pendleton added a comment - I think that should be all we need to do, but I am not certain. Can you please try removing that section as an experiment, and maybe do a clean build and some tests, to see if you encounter any problems? In the meantime, I think that your existing patch is a clear improvement over the current behavior, so I have committed it. I suggest that we leave this issue open while you investigate whether the query definition can be removed from the properties file, or whether it is not as simple as that.
        Hide
        ASF subversion and git services added a comment -

        Commit 1745414 from Bryan Pendleton in branch 'code/trunk'
        [ https://svn.apache.org/r1745414 ]

        DERBY-3181: DatabaseMetaData.getBestRowIdentifier behavior with invalid scope

        This patch was contributed by Danoja Dias (danojadias at gmail dot com)

        The DatabaseMetaData.getBestRowIdentifier method takes a number of arguments,
        including "scope", which is supposed to be one of the enumerated values:
        bestRowTemporary, bestRowTransaction, and bestRowSession.

        If an invalid scope argument was passed to this method, Derby was returning
        a hard-coded "empty" row identifier, which was slightly different, in detail,
        to the row identifier that is returned for a valid scope argument.

        Since JDBC does not require that we return such a row identifier for an
        invalid scope argument, it seems cleaner and more useful to throw an
        exception with a message indicating that an invalid scope argument was
        passed. The caller can then correct their application to make the call with
        a valid scope argument, and will then receive a valid row identifier.

        Show
        ASF subversion and git services added a comment - Commit 1745414 from Bryan Pendleton in branch 'code/trunk' [ https://svn.apache.org/r1745414 ] DERBY-3181 : DatabaseMetaData.getBestRowIdentifier behavior with invalid scope This patch was contributed by Danoja Dias (danojadias at gmail dot com) The DatabaseMetaData.getBestRowIdentifier method takes a number of arguments, including "scope", which is supposed to be one of the enumerated values: bestRowTemporary, bestRowTransaction, and bestRowSession. If an invalid scope argument was passed to this method, Derby was returning a hard-coded "empty" row identifier, which was slightly different, in detail, to the row identifier that is returned for a valid scope argument. Since JDBC does not require that we return such a row identifier for an invalid scope argument, it seems cleaner and more useful to throw an exception with a message indicating that an invalid scope argument was passed. The caller can then correct their application to make the call with a valid scope argument, and will then receive a valid row identifier.
        Hide
        Danoja Dias added a comment -

        I think we should remove from line 887 to line 908 from org.apache.derby.impl.jdbc.metadata.properties. Is there any more to remove that query? If not I think we can do it as a part of this issue.

        Show
        Danoja Dias added a comment - I think we should remove from line 887 to line 908 from org.apache.derby.impl.jdbc.metadata.properties. Is there any more to remove that query? If not I think we can do it as a part of this issue.
        Hide
        Bryan Pendleton added a comment -

        It seems like we maybe don't need the "getBestRowIdentifierEmpty" query
        in org.apache.derby.impl.jdbc.metadata.properties anymore? I think it was
        only referenced by the call to getPreparedQuery() at line 2160 of
        EmbedDatabaseMetaData.java, which now throws the exception instead.

        I think that we could file a separate, new JIRA, which is linked to this bug,
        and which states that we could remove that system query from metadata.properties.

        Or, we could do that as part of this issue.

        Which way would be better?

        Show
        Bryan Pendleton added a comment - It seems like we maybe don't need the "getBestRowIdentifierEmpty" query in org.apache.derby.impl.jdbc.metadata.properties anymore? I think it was only referenced by the call to getPreparedQuery() at line 2160 of EmbedDatabaseMetaData.java, which now throws the exception instead. I think that we could file a separate, new JIRA, which is linked to this bug, and which states that we could remove that system query from metadata.properties. Or, we could do that as part of this issue. Which way would be better?
        Hide
        Bryan Pendleton added a comment -

        The patch looks great, thanks for adding the new message! I ran the new
        code with a 'sqle.printStackTrace' in the exception block and verified that
        the new message looks fine and has the expected argument value when
        it is thrown. I think we're good to go with this change; I'll do a bit more
        testing and commit the change later today.

        Show
        Bryan Pendleton added a comment - The patch looks great, thanks for adding the new message! I ran the new code with a 'sqle.printStackTrace' in the exception block and verified that the new message looks fine and has the expected argument value when it is thrown. I think we're good to go with this change; I'll do a bit more testing and commit the change later today.
        Hide
        Danoja Dias added a comment - - edited

        Hi Bryan,
        I did changes. all tests passed. diff file is attached.

        Thanks

        Show
        Danoja Dias added a comment - - edited Hi Bryan, I did changes. all tests passed. diff file is attached. Thanks
        Hide
        Bryan Pendleton added a comment -

        Hi Danoja, it is great to hear that the tests are working better now in your environment.

        I think that, to move ahead with this change, we need to improve our patch slightly
        so that we use a real Derby message for this error, rather than the hard-coded
        "Invalid scope" string in the experimental patch we've been testing.

        This is a good chance to learn how we add a new error message to Derby.

        So this means:

        1) Modify java/shared/org/apache/derby/shared/common/reference/SQLState.java
        to define a new SQL state value for this error condition. I think you can just
        follow the pattern that is already in that file; try going to around line 950 in
        that file, pick a new unused SQLState code (e.g., 42XAT, perhaps?), and give
        it a friendly name, like LANG_INVALID_ROWID_SCOPE

        2) Modify java/engine/org/apache/derby/loc/messages.xml to add your message text.
        Just go to the corresponding location in that file (that is, where your SQLState
        lines up with the other SQLStates in its range), and add a XML text block that
        provides a message about the invalid scope value.

        I think that our error message should take the scope value as an argument, so
        something like: "Invalid row identifier scope:

        {0}

        "

        3) Then, we want our test code in DatabaseMetaDataTest.java to do more than
        just catch-and-ignore the SQLException for invalid scope; we want it to check
        that it got the right message in this case, so we need the catch block to say
        assertSQLState( ...), like we do elsewhere in those tests (see around lines 3585, say)

        After I add a new message, I always do a complete rebuild by doing

        ant clobber cleanjars
        ant all

        Once you think you've got your proposed patch all tidied up, and you've checked
        that the tests are still passing, please post your proposed final patch for review.

        Thanks!

        Show
        Bryan Pendleton added a comment - Hi Danoja, it is great to hear that the tests are working better now in your environment. I think that, to move ahead with this change, we need to improve our patch slightly so that we use a real Derby message for this error, rather than the hard-coded "Invalid scope" string in the experimental patch we've been testing. This is a good chance to learn how we add a new error message to Derby. So this means: 1) Modify java/shared/org/apache/derby/shared/common/reference/SQLState.java to define a new SQL state value for this error condition. I think you can just follow the pattern that is already in that file; try going to around line 950 in that file, pick a new unused SQLState code (e.g., 42XAT, perhaps?), and give it a friendly name, like LANG_INVALID_ROWID_SCOPE 2) Modify java/engine/org/apache/derby/loc/messages.xml to add your message text. Just go to the corresponding location in that file (that is, where your SQLState lines up with the other SQLStates in its range), and add a XML text block that provides a message about the invalid scope value. I think that our error message should take the scope value as an argument, so something like: "Invalid row identifier scope: {0} " 3) Then, we want our test code in DatabaseMetaDataTest.java to do more than just catch-and-ignore the SQLException for invalid scope; we want it to check that it got the right message in this case, so we need the catch block to say assertSQLState( ...), like we do elsewhere in those tests (see around lines 3585, say) After I add a new message, I always do a complete rebuild by doing ant clobber cleanjars ant all Once you think you've got your proposed patch all tidied up, and you've checked that the tests are still passing, please post your proposed final patch for review. Thanks!
        Hide
        Danoja Dias added a comment -

        Hi Bryan,
        I ran the test again after updating the repositary. Then no failures found.
        Test result is like this.
        junit-core:
        [junit] Running org.apache.derbyTesting.junit.EnvTest
        [junit] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.314 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.derbynet._Suite
        [junit] Tests run: 339, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 565.378 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.tools._Suite
        [junit] Tests run: 127, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 202.549 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.demo._Suite
        [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.831 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.lang._Suite
        [junit] Tests run: 3226, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9,402.799 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite
        [junit] Tests run: 7881, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3,496.768 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.store._Suite
        [junit] Tests run: 346, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2,802.678 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.engine._Suite
        [junit] Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 202.396 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite
        [junit] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,391.063 sec
        [junit] Running org.apache.derbyTesting.unitTests.junit._Suite
        [junit] Tests run: 170, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.206 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite
        [junit] Tests run: 6439, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8,327.352 sec
        [junit] Running org.apache.derbyTesting.functionTests.suites.EncryptionSuite
        [junit] Tests run: 203, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 203.698 sec

        Can we do the commits and change the status to resolved?

        Show
        Danoja Dias added a comment - Hi Bryan, I ran the test again after updating the repositary. Then no failures found. Test result is like this. junit-core: [junit] Running org.apache.derbyTesting.junit.EnvTest [junit] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.314 sec [junit] Running org.apache.derbyTesting.functionTests.tests.derbynet._Suite [junit] Tests run: 339, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 565.378 sec [junit] Running org.apache.derbyTesting.functionTests.tests.tools._Suite [junit] Tests run: 127, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 202.549 sec [junit] Running org.apache.derbyTesting.functionTests.tests.demo._Suite [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.831 sec [junit] Running org.apache.derbyTesting.functionTests.tests.lang._Suite [junit] Tests run: 3226, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9,402.799 sec [junit] Running org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite [junit] Tests run: 7881, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3,496.768 sec [junit] Running org.apache.derbyTesting.functionTests.tests.store._Suite [junit] Tests run: 346, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2,802.678 sec [junit] Running org.apache.derbyTesting.functionTests.tests.engine._Suite [junit] Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 202.396 sec [junit] Running org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite [junit] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,391.063 sec [junit] Running org.apache.derbyTesting.unitTests.junit._Suite [junit] Tests run: 170, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.206 sec [junit] Running org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite [junit] Tests run: 6439, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8,327.352 sec [junit] Running org.apache.derbyTesting.functionTests.suites.EncryptionSuite [junit] Tests run: 203, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 203.698 sec Can we do the commits and change the status to resolved?
        Hide
        Danoja Dias added a comment -

        Hi Bryan,

        I am getting many failures like following. I have not added all of them here. Why do they differ from you?

        <testcase classname="org.apache.derbyTesting.functionTests.tests.upgradeTests.BasicSetup" name="testOldVersion" time="0.035">
        <failure message="Old minor (driver): expected:<1> but was:<13>" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Old minor (driver): expected:<1> but was:<13>

        <testcase classname="org.apache.derbyTesting.functionTests.tests.upgradeTests.Changes10_2" name="testDatabaseOwnerChange" time="0.042">
        <failure message="AUTHORIZATIONID not valid for SYSIBM expected:<[DBA]> but was:<[APP]>" type="junit.framework.ComparisonFailure">junit.framework.ComparisonFailure: AUTHORIZATIONID not valid for SYSIBM expected:<[DBA]> but was:<[APP]>

        <testcase classname="org.apache.derbyTesting.functionTests.tests.upgradeTests.Changes10_4" name="testErrorMessage" time="0.327">
        <failure message="expected error while creating unique constraint over nullable column" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: expected error while creating unique constraint over nullable column

        Show
        Danoja Dias added a comment - Hi Bryan, I am getting many failures like following. I have not added all of them here. Why do they differ from you? <testcase classname="org.apache.derbyTesting.functionTests.tests.upgradeTests.BasicSetup" name="testOldVersion" time="0.035"> <failure message="Old minor (driver): expected:<1> but was:<13>" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Old minor (driver): expected:<1> but was:<13> <testcase classname="org.apache.derbyTesting.functionTests.tests.upgradeTests.Changes10_2" name="testDatabaseOwnerChange" time="0.042"> <failure message="AUTHORIZATIONID not valid for SYSIBM expected:< [DBA] > but was:< [APP] >" type="junit.framework.ComparisonFailure">junit.framework.ComparisonFailure: AUTHORIZATIONID not valid for SYSIBM expected:< [DBA] > but was:< [APP] > <testcase classname="org.apache.derbyTesting.functionTests.tests.upgradeTests.Changes10_4" name="testErrorMessage" time="0.327"> <failure message="expected error while creating unique constraint over nullable column" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: expected error while creating unique constraint over nullable column
        Hide
        Bryan Pendleton added a comment -

        Hi Danoja, it appears that my test failures are actually caused by DERBY-6856, and are not related to our work on this issue. If you are
        getting "Unexpected type DATE" in your upgrade tests, it is because
        of DERBY-6856. I will work on that problem separately.

        Show
        Bryan Pendleton added a comment - Hi Danoja, it appears that my test failures are actually caused by DERBY-6856 , and are not related to our work on this issue. If you are getting "Unexpected type DATE" in your upgrade tests, it is because of DERBY-6856 . I will work on that problem separately.
        Hide
        Bryan Pendleton added a comment -

        Indeed, it looks like I am still seeing some problems in the upgradeTests, too.
        But my results are not the same as yours. Can you look in your results?
        You should have a directory named "junit_YYYYMMDD_HHMM" in your trunk
        directory, and in there should be all the results of your testing. There
        should be a file named TEST-org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite.xml
        and that file should have information about each test failure. In that
        file, use your text editor to search for the strings 'error message=' and
        'failure message=' to see information about each error and each test failure.

        In my case, my results were:
        [junit] Tests run: 6439, Failures: 14, Errors: 0, Skipped: 0, Time elapsed: 2,491.535 sec
        [junit] Test org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite FAILED

        and in my XML file I see only one failure, but the same failure occurred
        all 14 times:
        <testcase classname="org.apache.derbyTesting.functionTests.tests.jdbcapi.DatabaseMetaDataTest" name="testGetTypeInfo" time="0.147">
        <failure message="Unexpected type DATE" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Unexpected type DATE
        at org.apache.derbyTesting.functionTests.tests.jdbcapi.DatabaseMetaDataTest.testGetTypeInfo(DatabaseMetaDataTest.java:2346)
        at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:120)
        ...

        I will have to study this some more; I am not sure what is causing this
        particular test to fail, and why it occurs only during the upgrade testing.

        What do your errors and failures look like?+

        Show
        Bryan Pendleton added a comment - Indeed, it looks like I am still seeing some problems in the upgradeTests, too. But my results are not the same as yours. Can you look in your results? You should have a directory named "junit_YYYYMMDD_HHMM" in your trunk directory, and in there should be all the results of your testing. There should be a file named TEST-org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite.xml and that file should have information about each test failure. In that file, use your text editor to search for the strings 'error message=' and 'failure message=' to see information about each error and each test failure. In my case, my results were: [junit] Tests run: 6439, Failures: 14, Errors: 0, Skipped: 0, Time elapsed: 2,491.535 sec [junit] Test org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite FAILED and in my XML file I see only one failure, but the same failure occurred all 14 times: <testcase classname="org.apache.derbyTesting.functionTests.tests.jdbcapi.DatabaseMetaDataTest" name="testGetTypeInfo" time="0.147"> <failure message="Unexpected type DATE" type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Unexpected type DATE at org.apache.derbyTesting.functionTests.tests.jdbcapi.DatabaseMetaDataTest.testGetTypeInfo(DatabaseMetaDataTest.java:2346) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:120) ... I will have to study this some more; I am not sure what is causing this particular test to fail, and why it occurs only during the upgrade testing. What do your errors and failures look like?+
        Hide
        Danoja Dias added a comment -

        Hi Bryan,

        Are they running clean?
        But I got something like this after adding the patch.

        junit-core:
        [junit] Running org.apache.derbyTesting.junit.EnvTest
        [junit] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.987 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.derbynet._Suite
        [junit] Tests run: 339, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 863.906 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.tools._Suite
        [junit] Tests run: 127, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 200.987 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.demo._Suite
        [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.368 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.lang._Suite
        [junit] Tests run: 3226, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11,455.151 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite
        [junit] Tests run: 7881, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5,330.829 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.store._Suite
        [junit] Tests run: 346, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3,127.492 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.engine._Suite
        [junit] Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 197.515 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite
        [junit] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,306.592 sec
        [junit] Running org.apache.derbyTesting.unitTests.junit._Suite
        [junit] Tests run: 170, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.173 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite
        [junit] Tests run: 6439, Failures: 152, Errors: 35, Skipped: 0, Time elapsed: 8,246.18 sec
        [junit] Test org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite FAILED
        [junit] Running org.apache.derbyTesting.functionTests.suites.EncryptionSuite
        [junit] Tests run: 203, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 199.743 sec

        Show
        Danoja Dias added a comment - Hi Bryan, Are they running clean? But I got something like this after adding the patch. junit-core: [junit] Running org.apache.derbyTesting.junit.EnvTest [junit] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.987 sec [junit] Running org.apache.derbyTesting.functionTests.tests.derbynet._Suite [junit] Tests run: 339, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 863.906 sec [junit] Running org.apache.derbyTesting.functionTests.tests.tools._Suite [junit] Tests run: 127, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 200.987 sec [junit] Running org.apache.derbyTesting.functionTests.tests.demo._Suite [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.368 sec [junit] Running org.apache.derbyTesting.functionTests.tests.lang._Suite [junit] Tests run: 3226, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11,455.151 sec [junit] Running org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite [junit] Tests run: 7881, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5,330.829 sec [junit] Running org.apache.derbyTesting.functionTests.tests.store._Suite [junit] Tests run: 346, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3,127.492 sec [junit] Running org.apache.derbyTesting.functionTests.tests.engine._Suite [junit] Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 197.515 sec [junit] Running org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite [junit] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,306.592 sec [junit] Running org.apache.derbyTesting.unitTests.junit._Suite [junit] Tests run: 170, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.173 sec [junit] Running org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite [junit] Tests run: 6439, Failures: 152, Errors: 35, Skipped: 0, Time elapsed: 8,246.18 sec [junit] Test org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite FAILED [junit] Running org.apache.derbyTesting.functionTests.suites.EncryptionSuite [junit] Tests run: 203, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 199.743 sec
        Hide
        Bryan Pendleton added a comment -

        With your change applied, I ran 'ant junit-core' and saw some test diffs:

        junit-core:
        [junit] Running org.apache.derbyTesting.junit.EnvTest
        [junit] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.296 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.derbynet._Suite
        [junit] Tests run: 330, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.169 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.tools._Suite
        [junit] Tests run: 127, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 92.221 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.demo._Suite
        [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.37 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.lang._Suite
        [junit] Tests run: 3225, Failures: 0, Errors: 14, Skipped: 0, Time elapsed: 2,976.494 sec
        [junit] Test org.apache.derbyTesting.functionTests.tests.lang._Suite FAILED
        [junit] Running org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite
        [junit] Tests run: 7881, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 1,526.023 sec
        [junit] Test org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite FAILED
        [junit] Running org.apache.derbyTesting.functionTests.tests.store._Suite
        [junit] Tests run: 346, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,026.574 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.engine._Suite
        [junit] Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 96.724 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite
        [junit] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 635.49 sec
        [junit] Running org.apache.derbyTesting.unitTests.junit._Suite
        [junit] Tests run: 166, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.923 sec
        [junit] Running org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite
        [junit] Tests run: 6439, Failures: 14, Errors: 138, Skipped: 0, Time elapsed: 2,170.677 sec
        [junit] Test org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite FAILED
        [junit] Running org.apache.derbyTesting.functionTests.suites.EncryptionSuite
        [junit] Tests run: 203, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.194 sec

        But when I looked at the actual test failures, it turned out that it was
        just the same test, jdbcapi/DatabaseMetaDataTest.java, which was
        failing – the test suites run that test multiple times, in multiple different
        configurations.

        DatabaseMetaDataTest calls getBestRowIdentifier with an invalid
        scope, intentionally, in order to test it, but was not expecting an exception.

        I changed the test to catch the expected exception for the invalid scope
        (see the testChange.diff patch), and now the tests are running clean.

        Can you try applying the testChange.diff patch to your system and
        see if the test results are improved for you?

        Show
        Bryan Pendleton added a comment - With your change applied, I ran 'ant junit-core' and saw some test diffs: junit-core: [junit] Running org.apache.derbyTesting.junit.EnvTest [junit] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.296 sec [junit] Running org.apache.derbyTesting.functionTests.tests.derbynet._Suite [junit] Tests run: 330, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.169 sec [junit] Running org.apache.derbyTesting.functionTests.tests.tools._Suite [junit] Tests run: 127, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 92.221 sec [junit] Running org.apache.derbyTesting.functionTests.tests.demo._Suite [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.37 sec [junit] Running org.apache.derbyTesting.functionTests.tests.lang._Suite [junit] Tests run: 3225, Failures: 0, Errors: 14, Skipped: 0, Time elapsed: 2,976.494 sec [junit] Test org.apache.derbyTesting.functionTests.tests.lang._Suite FAILED [junit] Running org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite [junit] Tests run: 7881, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 1,526.023 sec [junit] Test org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite FAILED [junit] Running org.apache.derbyTesting.functionTests.tests.store._Suite [junit] Tests run: 346, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,026.574 sec [junit] Running org.apache.derbyTesting.functionTests.tests.engine._Suite [junit] Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 96.724 sec [junit] Running org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite [junit] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 635.49 sec [junit] Running org.apache.derbyTesting.unitTests.junit._Suite [junit] Tests run: 166, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.923 sec [junit] Running org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite [junit] Tests run: 6439, Failures: 14, Errors: 138, Skipped: 0, Time elapsed: 2,170.677 sec [junit] Test org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite FAILED [junit] Running org.apache.derbyTesting.functionTests.suites.EncryptionSuite [junit] Tests run: 203, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.194 sec But when I looked at the actual test failures, it turned out that it was just the same test, jdbcapi/DatabaseMetaDataTest.java, which was failing – the test suites run that test multiple times, in multiple different configurations. DatabaseMetaDataTest calls getBestRowIdentifier with an invalid scope, intentionally, in order to test it, but was not expecting an exception. I changed the test to catch the expected exception for the invalid scope (see the testChange.diff patch), and now the tests are running clean. Can you try applying the testChange.diff patch to your system and see if the test results are improved for you?
        Hide
        Danoja Dias added a comment -

        Hi Bryan,

        It failed here. Still running. What happened in yours?

        Show
        Danoja Dias added a comment - Hi Bryan, It failed here. Still running. What happened in yours?
        Hide
        Bryan Pendleton added a comment -

        This looks great. I will run some tests on my system,
        and let you know what I see, so you can compare to your results.

        Show
        Bryan Pendleton added a comment - This looks great. I will run some tests on my system, and let you know what I see, so you can compare to your results.
        Hide
        Danoja Dias added a comment -

        Hi Bryan,

        I attached the diff file.
        I changed to throw an exception when scope argument is invalid. I think this is enough to check whether there is any depend on being able to call getBestRowIdentifier with an invalid scope value.

        Should I improve the way it throws the exception.

        Show
        Danoja Dias added a comment - Hi Bryan, I attached the diff file. I changed to throw an exception when scope argument is invalid. I think this is enough to check whether there is any depend on being able to call getBestRowIdentifier with an invalid scope value. Should I improve the way it throws the exception.
        Hide
        Bryan Pendleton added a comment -

        I had completely misunderstood that this job involved the behavior when
        the "scope" argument was invalid – thanks for pointing that out!

        Frankly, I don't understand why we are returning a ResultSet at all
        when the scope is invalid. Why not throw a SQLException instead?

        I looked through the Subversion history, and that behavior of
        getBestRowIdentifier is present in the original version of EmbedDatabaseMetaData.java
        in the original contribution from IBM.

        So this is very old behavior.

        It would be interesting, though, to try this experiment:
        1) Try changing EmbedDatabaseMetaData.getBestRowIdentifier to
        throw a SQLException if the scope is out of range, rather than
        executing getBestRowIdentifierEmpty
        2) Run all the regression tests (ant junit-core) and see if any
        of the regression tests fails due to this change

        That might give us some clues about which sorts of applications, if any,
        depend on being able to call getBestRowIdentifier with an invalid scope value.

        Show
        Bryan Pendleton added a comment - I had completely misunderstood that this job involved the behavior when the "scope" argument was invalid – thanks for pointing that out! Frankly, I don't understand why we are returning a ResultSet at all when the scope is invalid. Why not throw a SQLException instead? I looked through the Subversion history, and that behavior of getBestRowIdentifier is present in the original version of EmbedDatabaseMetaData.java in the original contribution from IBM. So this is very old behavior. It would be interesting, though, to try this experiment: 1) Try changing EmbedDatabaseMetaData.getBestRowIdentifier to throw a SQLException if the scope is out of range, rather than executing getBestRowIdentifierEmpty 2) Run all the regression tests (ant junit-core) and see if any of the regression tests fails due to this change That might give us some clues about which sorts of applications, if any, depend on being able to call getBestRowIdentifier with an invalid scope value.
        Hide
        Danoja Dias added a comment -

        In the first case of repro.java
        it runs following query for getBestRowIdentifier method,
        getBestRowIdentifierAllColumns=\
        SELECT \
        CAST (java.sql.DatabaseMetaData::bestRowSession AS SMALLINT) AS SCOPE, \
        COLS.COLUMNNAME AS COLUMN_NAME, \
        COLS.COLUMNDATATYPE.getJDBCTypeId() AS DATA_TYPE, \
        CAST (COLS.COLUMNDATATYPE.getTypeName() AS VARCHAR(128)) AS TYPE_NAME, \
        COLS.COLUMNDATATYPE.getMaximumWidth() AS COLUMN_SIZE, \
        CAST (NULL AS INT) AS BUFFER_LENGTH, \
        CAST ((CASE WHEN (COLS.COLUMNDATATYPE.getJDBCTypeId() IN ( \
        java.sql.Types::DECIMAL, java.sql.Types::NUMERIC, \
        java.sql.Types::INTEGER, java.sql.Types::SMALLINT, \
        java.sql.Types::TINYINT, java.sql.Types::BIGINT, \
        java.sql.Types::DATE, java.sql.Types::TIME, \
        java.sql.Types::TIMESTAMP)) \
        THEN COLS.COLUMNDATATYPE.getPrecision() \
        ELSE CAST (NULL AS SMALLINT) END) AS SMALLINT) \
        AS DECIMAL_DIGITS, \
        CAST (java.sql.DatabaseMetaData::bestRowNotPseudo AS SMALLINT) AS PSEUDO_COLUMN \
        FROM SYS.SYSSCHEMAS SCHEMAS, SYS.SYSTABLES TABS, \
        SYS.SYSCOLUMNS COLS \
        WHERE COLS.REFERENCEID = TABS.TABLEID \
        AND TABS.SCHEMAID = SCHEMAS.SCHEMAID \
        AND ((1=1) OR % IS NOT NULL) \
        AND (SCHEMAS.SCHEMANAME LIKE APP) \
        AND (TABS.TABLENAME=a) \
        AND 1 BETWEEN 0 AND 2 \
        AND (1<>0 OR NOT COLS.COLUMNDATATYPE.isNullable())

        Show
        Danoja Dias added a comment - In the first case of repro.java it runs following query for getBestRowIdentifier method, getBestRowIdentifierAllColumns=\ SELECT \ CAST (java.sql.DatabaseMetaData::bestRowSession AS SMALLINT) AS SCOPE, \ COLS.COLUMNNAME AS COLUMN_NAME, \ COLS.COLUMNDATATYPE.getJDBCTypeId() AS DATA_TYPE, \ CAST (COLS.COLUMNDATATYPE.getTypeName() AS VARCHAR(128)) AS TYPE_NAME, \ COLS.COLUMNDATATYPE.getMaximumWidth() AS COLUMN_SIZE, \ CAST (NULL AS INT) AS BUFFER_LENGTH, \ CAST ((CASE WHEN (COLS.COLUMNDATATYPE.getJDBCTypeId() IN ( \ java.sql.Types::DECIMAL, java.sql.Types::NUMERIC, \ java.sql.Types::INTEGER, java.sql.Types::SMALLINT, \ java.sql.Types::TINYINT, java.sql.Types::BIGINT, \ java.sql.Types::DATE, java.sql.Types::TIME, \ java.sql.Types::TIMESTAMP)) \ THEN COLS.COLUMNDATATYPE.getPrecision() \ ELSE CAST (NULL AS SMALLINT) END) AS SMALLINT) \ AS DECIMAL_DIGITS, \ CAST (java.sql.DatabaseMetaData::bestRowNotPseudo AS SMALLINT) AS PSEUDO_COLUMN \ FROM SYS.SYSSCHEMAS SCHEMAS, SYS.SYSTABLES TABS, \ SYS.SYSCOLUMNS COLS \ WHERE COLS.REFERENCEID = TABS.TABLEID \ AND TABS.SCHEMAID = SCHEMAS.SCHEMAID \ AND ((1=1) OR % IS NOT NULL) \ AND (SCHEMAS.SCHEMANAME LIKE APP) \ AND (TABS.TABLENAME=a) \ AND 1 BETWEEN 0 AND 2 \ AND (1<>0 OR NOT COLS.COLUMNDATATYPE.isNullable())
        Hide
        Danoja Dias added a comment -

        the argument scope of getBestRowIdentifier(String catalog,String schema,String table,int scope,boolean nullable) method must be 0 or 1 or 2. If not it is not a valid argument and It returns an empty resultset.
        In the latter part of repro.java, scope is 3 and it execute the following prepared query in metadata.properties file.
        getBestRowIdentifierEmpty=\
        SELECT SCOPE, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, \
        BUFFER_LENGTH, DECIMAL_DIGITS, PSEUDO_COLUMN \
        FROM (VALUES \
        (CAST (2 AS SMALLINT), \
        CAST ('' AS VARCHAR(128)), \
        0, \
        CAST ('INT' AS VARCHAR(128)), \
        0, \
        0, \
        CAST (0 AS SMALLINT), \
        CAST (0 AS SMALLINT)) \
        ) AS BESTROWIDENTIFIER( \
        SCOPE, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, \
        DECIMAL_DIGITS, PSEUDO_COLUMN) \
        WHERE (1=0)

        I think the bug occurs according to the resultset thst returns here.

        Show
        Danoja Dias added a comment - the argument scope of getBestRowIdentifier(String catalog,String schema,String table,int scope,boolean nullable) method must be 0 or 1 or 2. If not it is not a valid argument and It returns an empty resultset. In the latter part of repro.java, scope is 3 and it execute the following prepared query in metadata.properties file. getBestRowIdentifierEmpty=\ SELECT SCOPE, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, \ BUFFER_LENGTH, DECIMAL_DIGITS, PSEUDO_COLUMN \ FROM (VALUES \ (CAST (2 AS SMALLINT), \ CAST ('' AS VARCHAR(128)), \ 0, \ CAST ('INT' AS VARCHAR(128)), \ 0, \ 0, \ CAST (0 AS SMALLINT), \ CAST (0 AS SMALLINT)) \ ) AS BESTROWIDENTIFIER( \ SCOPE, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, \ DECIMAL_DIGITS, PSEUDO_COLUMN) \ WHERE (1=0) I think the bug occurs according to the resultset thst returns here.
        Hide
        Kathey Marsden added a comment -

        Triaged for 10.5.2. Set urgency to normal

        Show
        Kathey Marsden added a comment - Triaged for 10.5.2. Set urgency to normal

          People

          • Assignee:
            Danoja Dias
            Reporter:
            Myrna van Lunteren
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development