Derby
  1. Derby
  2. DERBY-5966

NativeAuthenticationServiceTest.testAll() failure: [ FILE, NATIVE authentication on, LOCAL authentication ON, Authentication/Authorization turned OFF, SecurityManager ON, Embedded ]: Connection to fifthDB unexpectedly failed.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 10.10.1.1
    • Fix Version/s: 10.10.1.1
    • Component/s: Test
    • Labels:
      None

      Description

      Saw these failures on multiple platforms around the time Europe switched from DST:

      http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.5/testing/testlog/sles/1402814-suitesAll_diff.txt
      http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.5/testing/testlog/sparc/1402814-suitesAll_diff.txt
      http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.5/testing/testlog/vista/1402814-suitesAll_diff.txt

      3) testAll(org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest)junit.framework.AssertionFailedError: [ FILE, NATIVE authentication on, LOCAL authentication ON, Authentication/Authorization turned OFF, SecurityManager ON, Embedded ]: Connection to fifthDB unexpectedly failed.
      at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1604)
      at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1570)
      at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1564)
      at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.vetPasswordLifetime(NativeAuthenticationServiceTest.java:1385)
      at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.testAll(NativeAuthenticationServiceTest.java:618)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      1. d5966-1a-gmt.diff
        2 kB
        Knut Anders Hatlen

        Issue Links

          Activity

          Hide
          Knut Anders Hatlen added a comment -

          Attaching a patch that makes the test run in the GMT time zone for now. Let's revisit and see if we can get it to run in the local time zone once DERBY-5974 is fixed.

          Committed revision 1404888.

          Show
          Knut Anders Hatlen added a comment - Attaching a patch that makes the test run in the GMT time zone for now. Let's revisit and see if we can get it to run in the local time zone once DERBY-5974 is fixed. Committed revision 1404888.
          Hide
          Knut Anders Hatlen added a comment -

          Yes, I think that would be OK. We should probably also add a decorator around the test to silence it for future time zone changes until DERBY-4611 and DERBY-5974 have been fixed.

          Show
          Knut Anders Hatlen added a comment - Yes, I think that would be OK. We should probably also add a decorator around the test to silence it for future time zone changes until DERBY-4611 and DERBY-5974 have been fixed.
          Hide
          Rick Hillegas added a comment -

          I have logged DERBY-5974, an enhancement for converting all of the TIMESTAMP metadata columns to TIMESTAMP WITH TIMEZONE (after we implement that datatype). As part of that issue, I suspect that we would convert all TIMESTAMP metadata to GMT, including SYS.SYSUSERS.LASTMODIFIED. Until we make that change, I propose that we live with the defect tracked by this bug and address it with some weasel words in the Reference Manual sections on the password management procs. Does that seem sufficient? Thanks.

          Show
          Rick Hillegas added a comment - I have logged DERBY-5974 , an enhancement for converting all of the TIMESTAMP metadata columns to TIMESTAMP WITH TIMEZONE (after we implement that datatype). As part of that issue, I suspect that we would convert all TIMESTAMP metadata to GMT, including SYS.SYSUSERS.LASTMODIFIED. Until we make that change, I propose that we live with the defect tracked by this bug and address it with some weasel words in the Reference Manual sections on the password management procs. Does that seem sufficient? Thanks.
          Hide
          Knut Anders Hatlen added a comment -

          Option 3 might actually be a good first step for implementing DERBY-4611 (TIMESTAMP WITH TIMEZONE). Option 1 sounds like less work.

          Or we could just release note that already persisted passwords may have a few hours added to or removed from their lifetimes on upgrade, and skip the upgrade logic. Since the feature is so new, I wouldn't expect that many have started using it in ways that depend on the exact lifetime of the passwords, so hopefully that could be enough to manage any upgrade issues.

          Show
          Knut Anders Hatlen added a comment - Option 3 might actually be a good first step for implementing DERBY-4611 (TIMESTAMP WITH TIMEZONE). Option 1 sounds like less work. Or we could just release note that already persisted passwords may have a few hours added to or removed from their lifetimes on upgrade, and skip the upgrade logic. Since the feature is so new, I wouldn't expect that many have started using it in ways that depend on the exact lifetime of the passwords, so hopefully that could be enough to manage any upgrade issues.
          Hide
          Rick Hillegas added a comment -

          Thanks for that suggestion, Knut. I think it will work. However, it requires different (de)serialization logic for LASTMODIFIED stamps, depending on whether they were written to SYSUSERS before or after the fix is applied.

          1) One approach to this problem would be to overload HASHINGSCHEME so that it both describes the technique used to hash PASSWORD as well as the technique used to serialize LASTMODIFIED. We could introduce a new prefix for the HASHINGSCHEME column: ID_PATTERN_CONFIGURABLE_STRETCHED_SCHEME_GMT. This prefix would imply hashing behavior identical to ID_PATTERN_CONFIGURABLE_STRETCHED_SCHEME but it would encode the fact that LASTMODIFIED was shifted to GMT before being serialized.

          2) Another approach would be to add a timezone column to SYS.SYSUSERS.

          3) Another approach would be to add a time-zone aware variant of SQLTIMESTAMP and store that in LASTMODIFIED.

          Other suggestions?

          Thanks,
          -Rick

          Show
          Rick Hillegas added a comment - Thanks for that suggestion, Knut. I think it will work. However, it requires different (de)serialization logic for LASTMODIFIED stamps, depending on whether they were written to SYSUSERS before or after the fix is applied. 1) One approach to this problem would be to overload HASHINGSCHEME so that it both describes the technique used to hash PASSWORD as well as the technique used to serialize LASTMODIFIED. We could introduce a new prefix for the HASHINGSCHEME column: ID_PATTERN_CONFIGURABLE_STRETCHED_SCHEME_GMT. This prefix would imply hashing behavior identical to ID_PATTERN_CONFIGURABLE_STRETCHED_SCHEME but it would encode the fact that LASTMODIFIED was shifted to GMT before being serialized. 2) Another approach would be to add a timezone column to SYS.SYSUSERS. 3) Another approach would be to add a time-zone aware variant of SQLTIMESTAMP and store that in LASTMODIFIED. Other suggestions? Thanks, -Rick
          Hide
          Knut Anders Hatlen added a comment -

          An alternative may be to store the timestamp in GMT, which would not be affected by time zone changes.

          It might be as simple as changing this code in SYSUSERSRowFactory.makeRow() from

          /* 4th column is LASTMODIFIED (timestamp) */
          row.setColumn( LASTMODIFIED_COL_NUM, new SQLTimestamp( lastModified ) );

          to

          /* 4th column is LASTMODIFIED (timestamp) */
          SQLTimestamp universalModTime = new SQLTimestamp();
          universalModTime.setValue(lastModified, Calendar.getInstance(TimeZone.getTimeZone("GMT")));
          row.setColumn( LASTMODIFIED_COL_NUM, universalModTime);

          and in buildDescriptor() from

          /* 4th column is LASTMODIFIED */
          col = row.getColumn( LASTMODIFIED_COL_NUM );
          lastModified = col.getTimestamp( new java.util.GregorianCalendar() );

          to

          /* 4th column is LASTMODIFIED */
          col = row.getColumn( LASTMODIFIED_COL_NUM );
          lastModified = col.getTimestamp(Calendar.getInstance(TimeZone.getTimeZone("GMT")));

          Show
          Knut Anders Hatlen added a comment - An alternative may be to store the timestamp in GMT, which would not be affected by time zone changes. It might be as simple as changing this code in SYSUSERSRowFactory.makeRow() from /* 4th column is LASTMODIFIED (timestamp) */ row.setColumn( LASTMODIFIED_COL_NUM, new SQLTimestamp( lastModified ) ); to /* 4th column is LASTMODIFIED (timestamp) */ SQLTimestamp universalModTime = new SQLTimestamp(); universalModTime.setValue(lastModified, Calendar.getInstance(TimeZone.getTimeZone("GMT"))); row.setColumn( LASTMODIFIED_COL_NUM, universalModTime); and in buildDescriptor() from /* 4th column is LASTMODIFIED */ col = row.getColumn( LASTMODIFIED_COL_NUM ); lastModified = col.getTimestamp( new java.util.GregorianCalendar() ); to /* 4th column is LASTMODIFIED */ col = row.getColumn( LASTMODIFIED_COL_NUM ); lastModified = col.getTimestamp(Calendar.getInstance(TimeZone.getTimeZone("GMT")));
          Hide
          Rick Hillegas added a comment -

          I am reluctant to change the stored format of Derby TIMESTAMPs. I am also reluctant to build complicated timezone-aware machinery to police this problem. I am inclined to add a warning to the documentation for SYSCS_UTIL.SYSCS_CREATE_USER, SYSCS_UTIL.SYSCS_MODIFY_PASSWORD, and SYSCS_UTIL.SYSCS_RESET_PASSWORD: password lifetimes may be skewed by timezone changes.

          Show
          Rick Hillegas added a comment - I am reluctant to change the stored format of Derby TIMESTAMPs. I am also reluctant to build complicated timezone-aware machinery to police this problem. I am inclined to add a warning to the documentation for SYSCS_UTIL.SYSCS_CREATE_USER, SYSCS_UTIL.SYSCS_MODIFY_PASSWORD, and SYSCS_UTIL.SYSCS_RESET_PASSWORD: password lifetimes may be skewed by timezone changes.
          Hide
          Knut Anders Hatlen added a comment -

          That sounds right. I'd guess it's similar to DERBY-5964, where a timestamp is stored in the local time zone in the database. Since Derby doesn't store the time zone alongside the timestamp, the timestamps are ambiguous around the time the time zone changes, and they may come out one hour off when they are read from the database.

          I suppose that in most cases it doesn't matter much if a password expires one hour too soon or too late, as the password lifetime would be significantly longer than that. However, derby.authentication.native.passwordLifetimeMillis allows specification of password expiry with millisecond granularity, so one cannot completely rule out the possibility that some applications use sub-hour password expiry.

          Show
          Knut Anders Hatlen added a comment - That sounds right. I'd guess it's similar to DERBY-5964 , where a timestamp is stored in the local time zone in the database. Since Derby doesn't store the time zone alongside the timestamp, the timestamps are ambiguous around the time the time zone changes, and they may come out one hour off when they are read from the database. I suppose that in most cases it doesn't matter much if a password expires one hour too soon or too late, as the password lifetime would be significantly longer than that. However, derby.authentication.native.passwordLifetimeMillis allows specification of password expiry with millisecond granularity, so one cannot completely rule out the possibility that some applications use sub-hour password expiry.
          Hide
          Rick Hillegas added a comment -

          The error in question managed to print a complete stack trace for the problem to *suitesAll.diff.txt:

          (emb)lang.NativeAuthenticationServiceTest.testAll junit.framework.AssertionFailedError: [ FILE, NATIVE authentication on, LOCAL authentication ON, Authentication/Authorization turned OFF, SecurityManager ON, Embedded ]: Connection to fifthDB should have failed.
          at junit.framework.Assert.fail(Assert.java:47)
          at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1583)
          at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1570)
          at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1564)
          at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.vetPasswordLifetime(NativeAuthenticationServiceTest.java:1385)
          at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.testAll(NativeAuthenticationServiceTest.java:618)

          This indicates that the test was expecting a connection failure by a user whose password had expired. But the connection succeeded. I can see this situation arising if clocks change during the test run. There may be a product issue here but I am not sure how serious it is. Inside Derby, a comparison is being made between the current time returned by System.currentTimeMillis() and the contents of a SQL TIMESTAMP stored in SYS.SYSUSERS.LASTMODIFIED. My suspicion would be that the on-disk and in-memory representations of TIMESTAMPs have some interesting convertibility behavior during the twilit hour of overlap when the time zone changes.

          Show
          Rick Hillegas added a comment - The error in question managed to print a complete stack trace for the problem to *suitesAll.diff.txt: (emb)lang.NativeAuthenticationServiceTest.testAll junit.framework.AssertionFailedError: [ FILE, NATIVE authentication on, LOCAL authentication ON, Authentication/Authorization turned OFF, SecurityManager ON, Embedded ]: Connection to fifthDB should have failed. at junit.framework.Assert.fail(Assert.java:47) at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1583) at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1570) at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.getConnection(NativeAuthenticationServiceTest.java:1564) at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.vetPasswordLifetime(NativeAuthenticationServiceTest.java:1385) at org.apache.derbyTesting.functionTests.tests.lang.NativeAuthenticationServiceTest.testAll(NativeAuthenticationServiceTest.java:618) This indicates that the test was expecting a connection failure by a user whose password had expired. But the connection succeeded. I can see this situation arising if clocks change during the test run. There may be a product issue here but I am not sure how serious it is. Inside Derby, a comparison is being made between the current time returned by System.currentTimeMillis() and the contents of a SQL TIMESTAMP stored in SYS.SYSUSERS.LASTMODIFIED. My suspicion would be that the on-disk and in-memory representations of TIMESTAMPs have some interesting convertibility behavior during the twilit hour of overlap when the time zone changes.

            People

            • Assignee:
              Knut Anders Hatlen
              Reporter:
              Knut Anders Hatlen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development