Derby
  1. Derby
  2. DERBY-5622

Reduce the chance for hash collisions when checking bootPassword at boot time and when changing password.

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.8.3.0, 10.9.2.2, 10.10.1.1
    • Component/s: Store
    • Labels:
      None
    • Bug behavior facts:
      Data corruption, Security

      Description

      There are two issues, already seen in DERBY-2687:

      "the boot issue": there is a 1/2**16 chance that a wrong bootPassword will allow boot to proceed (but since its decoded key is wrong the boot will fail).
      "the password change" issue: similarly, there is a chance that the wrong bootPassword will be accepted trying to change it via
      SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY('bootPassword', ...) at least for algorithms that do not check IV (initialization vector) in addition to the
      digest, e.g. "DES/ECB/NoPadding"

      The latter case may lead to data corruption, cf. DERBY-2687 discussion. I think the risk is fairly low, though: One would need to have execution permission to change the property if SQL authorization is used, and in most scenarios the supplied existing password would be correct. But since the results can be bad, it would be good to reduce or eliminate the risk.

      1. repro.sh
        1 kB
        Dag H. Wanvik
      2. derby-5622-TT-fixWithTestScaffolding.diff
        4 kB
        Rick Hillegas
      3. derby-5622-repro.sql
        0.4 kB
        Rick Hillegas
      4. derby-5622-instrumentation.diff
        2 kB
        Dag H. Wanvik
      5. derby-5622-01-aa-decryptEncryptedSample.diff
        3 kB
        Rick Hillegas

        Issue Links

          Activity

          Gavin made changes -
          Workflow jira [ 12653821 ] Default workflow, editable Closed status [ 12796954 ]
          Kathey Marsden made changes -
          Fix Version/s 10.9.2.0 [ 12323562 ]
          Fix Version/s 10.9.1.1 [ 12321551 ]
          Kathey Marsden made changes -
          Fix Version/s 10.8.3.0 [ 12323456 ]
          Fix Version/s 10.8.2.3 [ 12318540 ]
          Rick Hillegas made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 10.8.2.3 [ 12318540 ]
          Fix Version/s 10.9.1.1 [ 12321551 ]
          Fix Version/s 10.10.0.0 [ 12321550 ]
          Resolution Fixed [ 1 ]
          Hide
          Rick Hillegas added a comment -

          Ported 1356749 from trunk to 10.9 at subversion revision 1363472.

          Ported 1356749 from trunk to 10.8 at subversion revision 1363475.

          Resolving this issue.

          Show
          Rick Hillegas added a comment - Ported 1356749 from trunk to 10.9 at subversion revision 1363472. Ported 1356749 from trunk to 10.8 at subversion revision 1363475. Resolving this issue.
          Hide
          Rick Hillegas added a comment -

          Thanks for verifying the patch, Dag. Committed derby-5622-01-aa-decryptEncryptedSample.diff at subversion revision 1356749.

          Show
          Rick Hillegas added a comment - Thanks for verifying the patch, Dag. Committed derby-5622-01-aa-decryptEncryptedSample.diff at subversion revision 1356749.
          Hide
          Dag H. Wanvik added a comment -

          I looked at the patch and verified that the second line of defense here kicks in if the first one is disabled as in your instrumented patch version.
          Patch looks good, +1 .

          Nits: some too long lines. One new line has a TAB:

          { throw StandardException.newException(SQLState.WRONG_BOOT_PASSWORD); }

          On that note, I'd prefer a newline after the "{" starting the explicit block.

          Show
          Dag H. Wanvik added a comment - I looked at the patch and verified that the second line of defense here kicks in if the first one is disabled as in your instrumented patch version. Patch looks good, +1 . Nits: some too long lines. One new line has a TAB: { throw StandardException.newException(SQLState.WRONG_BOOT_PASSWORD); } On that note, I'd prefer a newline after the "{" starting the explicit block.
          Hide
          Dag H. Wanvik added a comment -

          Souns like a good approach, Rick, thanks. Like you, I am at a loss as far as computing the total collision probability here, but your approach would seem to help. I'll have a look at the detailed patch tomorrow.
          I think I'd pass over any special bypass password, I'm happy with manual inspection and testing here.

          Show
          Dag H. Wanvik added a comment - Souns like a good approach, Rick, thanks. Like you, I am at a loss as far as computing the total collision probability here, but your approach would seem to help. I'll have a look at the detailed patch tomorrow. I think I'd pass over any special bypass password, I'm happy with manual inspection and testing here.
          Rick Hillegas made changes -
          Attachment derby-5622-01-aa-decryptEncryptedSample.diff [ 12531874 ]
          Attachment derby-5622-TT-fixWithTestScaffolding.diff [ 12531875 ]
          Attachment derby-5622-repro.sql [ 12531876 ]
          Hide
          Rick Hillegas added a comment -

          Attaching derby-5622-01-aa-decryptEncryptedSample.diff, derby-5622-TT-fixWithTestScaffolding.diff, and derby-5622-repro.sql. The first file is a patch for this issue, described below. The second file is the patch plus some extra edits which disable the pre-existing checks for whether the user has supplied the correct old bootpassword. The third file is a script demonstrating that a new check, added by the patch, runs if you make it past the pre-existing checks. To demonstrate the new behavior, apply derby-5622-TT-fixWithTestScaffolding.diff, run derby-5622-repro.sql, and look at the stack trace in derby.log to confirm that the new code was exercised.

          Regression tests passed cleanly for me after applying derby-5622-01-aa-decryptEncryptedSample.diff.

          Here is a re-cap of the current behavior in 10.8. Thanks to Dag and Knut for investigating this behavior here and on DERBY-2687:

          1) Let us say that the database was encrypted and booted using a bootpassword, which we will call $

          {actualBootPassword}.

          2) Note that when the database was created, Derby generated an encryption key, scrambled it using ${actualBootPassword}

          , and stored the scrambled result in service.properties. The generated encryption key is what is actually used to encrypt and decrypt data in the conglomerates and log files. In particular, the storage layer uses the encryption key to construct an object which encrypts data in the conglomerates. Let us call this object the storeEncrypter.

          3) The DBO can change the boot password by calling SYSCS_UTIL.SYSCS_SET_DATABASE_PASSWORD. If the key argument is "bootPassword", then Derby expects that the value argument will be a string of the form "${oldBootPassword,$

          {newBootPassword}", where ${oldBootPassword} is expected to be ${actualBootPassword}. Note the comma in the value argument string. The comma is important. It separates ${oldBootPassword} from ${newBootPassword}

          , effectively allowing the caller to pass 3 arguments to a procedure which expects two arguments. This peculiar syntax is discussed earlier on this issue and on DERBY-4328. Here is an example of how to call SYSCS_SET_DATABASE_PASSWORD to change the bootpassword:

          call syscs_util.syscs_set_database_property( 'bootPassword', 'bpass_02,bpass_03' );

          4) Before honoring this command, Derby needs to verify that the caller knows what they are doing. In particular, Derby needs to verify that $

          {oldBootPassword} really is ${actualBootPassword}. Presumably for security reasons, Derby doesn't stash ${actualBootPassword} anywhere, either on disk or in memory. Derby tries to verify the equivalence of ${oldBootPassword}

          and $

          {actualBootPassword}

          by first using $

          {oldBootPassword} to unscramble the encryption key stored in service.properties. Then Derby runs the following checks to verify that the unscrambed key is the actual encryption key used by storeEncrypter:

          i) Derby verifies that a 16 bit digest (appended to the scrambled result) is the same as a 16 bit digest computed from the unscrambled result. We believe that Derby more or less randomly generated the original encryption key. Therefore, the chance that a wrong ${oldBootPassword}

          will pass this check is smallish, viz., 1 in 2<sup>16</sup>.

          ii) If the previous check succeeds, then Derby performs a second check. Derby verifies that the unscrambled encryption key gives rise to the same initialization vector used by the storeEncrypter. An initialization vector is a seed for the encryption logic, as described here: http://en.wikipedia.org/wiki/Initialization_vector. I don't understand the combinatorics of constructing the initialization vector. So I can't estimate the risk that this check will succeed on a wrong $

          {oldBootPassword} after passing the check. However, the initialization vector is at most 128 bits long, so the probability of a collision is at least 1 in 2<sup>128</sup>.

          So the chance that a bad ${oldBootPassword}

          will pass both and (ii) is at least 1 in 2<sup>144</sup>.

          So far I have described the process of changing the bootpassword in the current Derby release, 10.8. The attached derby-5622-01-aa-decryptEncryptedSample.diff patch adds a third check in case and (ii) both succeed:

          (iii) We construct a candidateDecrypter from the unscrambled encryption key. Then we construct a stereotypical block of cleartext, which is 8192 bits long. We encrypt the cleartext using the storeEncrypter. Then we decrypt the result using the candidateDecrypter. If the final result matches the original cleartext, then the check passes. Again, I don't understand the combinatorics of Derby's encryption algorithms on this stereotypical cleartext, and I don't understand how they interact on keys which pass the and (ii) checks. However, the probability that a bad $

          {oldBootPassword} will pass this new check is at least 1 in 2<sup>8192</sup>.

          So the chance that a bad ${oldBootPassword}

          will pass , (ii), and (iii) is at least 1 in 2<sup>8336</sup>. That, however, is a very big number.

          I think that this additional check probably reduces the risk of this bug to an acceptable level, particularly if we advise people to backup their database before changing the bootpassword.

          I welcome your review as well as any suggestions about how to test this new codepath: I'm not sure how to coax Derby past and (ii) without instrumenting the code in a way which decreases security. But here is an idea:

          We could introduce a special $

          {oldBootPassword}

          , viz., "derby-5622-bypass". When Derby sees that value, it skips and (ii) and falls into check (iii). If somehow we still pass check (iii), then we raise an exception.

          Thanks in advance for your advice.

          Show
          Rick Hillegas added a comment - Attaching derby-5622-01-aa-decryptEncryptedSample.diff, derby-5622-TT-fixWithTestScaffolding.diff, and derby-5622-repro.sql. The first file is a patch for this issue, described below. The second file is the patch plus some extra edits which disable the pre-existing checks for whether the user has supplied the correct old bootpassword. The third file is a script demonstrating that a new check, added by the patch, runs if you make it past the pre-existing checks. To demonstrate the new behavior, apply derby-5622-TT-fixWithTestScaffolding.diff, run derby-5622-repro.sql, and look at the stack trace in derby.log to confirm that the new code was exercised. Regression tests passed cleanly for me after applying derby-5622-01-aa-decryptEncryptedSample.diff. Here is a re-cap of the current behavior in 10.8. Thanks to Dag and Knut for investigating this behavior here and on DERBY-2687 : 1) Let us say that the database was encrypted and booted using a bootpassword, which we will call $ {actualBootPassword}. 2) Note that when the database was created, Derby generated an encryption key, scrambled it using ${actualBootPassword} , and stored the scrambled result in service.properties. The generated encryption key is what is actually used to encrypt and decrypt data in the conglomerates and log files. In particular, the storage layer uses the encryption key to construct an object which encrypts data in the conglomerates. Let us call this object the storeEncrypter. 3) The DBO can change the boot password by calling SYSCS_UTIL.SYSCS_SET_DATABASE_PASSWORD. If the key argument is "bootPassword", then Derby expects that the value argument will be a string of the form "${oldBootPassword,$ {newBootPassword}", where ${oldBootPassword} is expected to be ${actualBootPassword}. Note the comma in the value argument string. The comma is important. It separates ${oldBootPassword} from ${newBootPassword} , effectively allowing the caller to pass 3 arguments to a procedure which expects two arguments. This peculiar syntax is discussed earlier on this issue and on DERBY-4328 . Here is an example of how to call SYSCS_SET_DATABASE_PASSWORD to change the bootpassword: call syscs_util.syscs_set_database_property( 'bootPassword', 'bpass_02,bpass_03' ); 4) Before honoring this command, Derby needs to verify that the caller knows what they are doing. In particular, Derby needs to verify that $ {oldBootPassword} really is ${actualBootPassword}. Presumably for security reasons, Derby doesn't stash ${actualBootPassword} anywhere, either on disk or in memory. Derby tries to verify the equivalence of ${oldBootPassword} and $ {actualBootPassword} by first using $ {oldBootPassword} to unscramble the encryption key stored in service.properties. Then Derby runs the following checks to verify that the unscrambed key is the actual encryption key used by storeEncrypter: i) Derby verifies that a 16 bit digest (appended to the scrambled result) is the same as a 16 bit digest computed from the unscrambled result. We believe that Derby more or less randomly generated the original encryption key. Therefore, the chance that a wrong ${oldBootPassword} will pass this check is smallish, viz., 1 in 2<sup>16</sup>. ii) If the previous check succeeds, then Derby performs a second check. Derby verifies that the unscrambled encryption key gives rise to the same initialization vector used by the storeEncrypter. An initialization vector is a seed for the encryption logic, as described here: http://en.wikipedia.org/wiki/Initialization_vector . I don't understand the combinatorics of constructing the initialization vector. So I can't estimate the risk that this check will succeed on a wrong $ {oldBootPassword} after passing the check. However, the initialization vector is at most 128 bits long, so the probability of a collision is at least 1 in 2<sup>128</sup>. So the chance that a bad ${oldBootPassword} will pass both and (ii) is at least 1 in 2<sup>144</sup>. So far I have described the process of changing the bootpassword in the current Derby release, 10.8. The attached derby-5622-01-aa-decryptEncryptedSample.diff patch adds a third check in case and (ii) both succeed: (iii) We construct a candidateDecrypter from the unscrambled encryption key. Then we construct a stereotypical block of cleartext, which is 8192 bits long. We encrypt the cleartext using the storeEncrypter. Then we decrypt the result using the candidateDecrypter. If the final result matches the original cleartext, then the check passes. Again, I don't understand the combinatorics of Derby's encryption algorithms on this stereotypical cleartext, and I don't understand how they interact on keys which pass the and (ii) checks. However, the probability that a bad $ {oldBootPassword} will pass this new check is at least 1 in 2<sup>8192</sup>. So the chance that a bad ${oldBootPassword} will pass , (ii), and (iii) is at least 1 in 2<sup>8336</sup>. That, however, is a very big number. I think that this additional check probably reduces the risk of this bug to an acceptable level, particularly if we advise people to backup their database before changing the bootpassword. I welcome your review as well as any suggestions about how to test this new codepath: I'm not sure how to coax Derby past and (ii) without instrumenting the code in a way which decreases security. But here is an idea: We could introduce a special $ {oldBootPassword} , viz., "derby-5622-bypass". When Derby sees that value, it skips and (ii) and falls into check (iii). If somehow we still pass check (iii), then we raise an exception. Thanks in advance for your advice.
          Kim Haase made changes -
          Link This issue is related to DERBY-5805 [ DERBY-5805 ]
          Hide
          Rick Hillegas added a comment -

          Thanks for the additional comments, Dag. I'm proposing that we introduce the new procedure as a separate, orthogonal issue.

          I agree that the bug still needs to be addressed. I'm convinced that there is value in just changing the boot password without re-encrypting the data. Right now, I'm considering two approaches to addressing this issue:

          1) Save the real boot password somewhere at boot time and compare it to what is passed into the password-changing procedure (currently SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY). The advantage of this solution is that it is airtight. The disadvantage is that a sensitive piece of information would be maintained in memory somewhere, which seems like a security vulnerability to me.

          2) Continue to try to reconstruct the generated key from the boot password that was passed to the password-changing procedure. Use this to create a new CipherProvider. Then use the old CipherProvider to encrypt some long byte array and decrypt the result with the new CipherProvider. If the result is the original byte array, then the boot password was probably correct. This is not airtight, but with a sufficiently long byte array the probability of data corruption might be vanishingly small.

          I would be interested in your thoughts. Thanks.

          Show
          Rick Hillegas added a comment - Thanks for the additional comments, Dag. I'm proposing that we introduce the new procedure as a separate, orthogonal issue. I agree that the bug still needs to be addressed. I'm convinced that there is value in just changing the boot password without re-encrypting the data. Right now, I'm considering two approaches to addressing this issue: 1) Save the real boot password somewhere at boot time and compare it to what is passed into the password-changing procedure (currently SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY). The advantage of this solution is that it is airtight. The disadvantage is that a sensitive piece of information would be maintained in memory somewhere, which seems like a security vulnerability to me. 2) Continue to try to reconstruct the generated key from the boot password that was passed to the password-changing procedure. Use this to create a new CipherProvider. Then use the old CipherProvider to encrypt some long byte array and decrypt the result with the new CipherProvider. If the result is the original byte array, then the boot password was probably correct. This is not airtight, but with a sufficiently long byte array the probability of data corruption might be vanishingly small. I would be interested in your thoughts. Thanks.
          Hide
          Dag H. Wanvik added a comment - - edited

          Note that the attribute called encryptedBootPassword is a bit of a misnomer: The value is actually the db encryption key encrypted using the boot password plus a hash of the db encryption key.

          Btw, Rick, even if we introduce a new method for this, the issue would remain, unless we change the behavior to generate a new encryption key and re-encrypt. Is that what you are proposing?

          Show
          Dag H. Wanvik added a comment - - edited Note that the attribute called encryptedBootPassword is a bit of a misnomer: The value is actually the db encryption key encrypted using the boot password plus a hash of the db encryption key. Btw, Rick, even if we introduce a new method for this, the issue would remain, unless we change the behavior to generate a new encryption key and re-encrypt. Is that what you are proposing?
          Hide
          Rick Hillegas added a comment -

          Hi Kim. Yes, I think it's worth clarifying the documentation. Thanks!

          Show
          Rick Hillegas added a comment - Hi Kim. Yes, I think it's worth clarifying the documentation. Thanks!
          Hide
          Kim Haase added a comment -

          You explain this with remarkable clarity, Rick – and I don't think we ever document clearly that using bootPassword is incompatible with using encryptionKey. Shall I file a doc issue on this?

          Show
          Kim Haase added a comment - You explain this with remarkable clarity, Rick – and I don't think we ever document clearly that using bootPassword is incompatible with using encryptionKey. Shall I file a doc issue on this?
          Hide
          Rick Hillegas added a comment -

          Thanks for the repro and the additional analysis, Dag. Here are some more observations:

          1) When you create an encrypted database, you specify either an explicit encryption key or a boot password. If you specify a boot password, then Derby generates an encryption key for you. You can't go back and forth between these techniques. Once you specify a boot password, you can't try to figure out the generated encryption key and boot the database using the encryption key. Similarly, if you create the database with an explicit encryption key, you can't try to introduce a boot password later on.

          This is because Derby stores different verification metadata for the two techniques. The verification metadata is used to catch bad boot passwords and encryption keys before actually trying to decrypt the conglomerates. If Derby can catch a bad boot password or encryption key before trying to decrypt the conglomerates, then Derby can raise an intelligible error message.

          If the database was created with a boot password, then Derby expects to find an encryptedBootPassword attribute in service.properties. The value of this attribute includes a digest used at boot time to verify the boot password before trying to decrypt the conglomerates.

          In contrast, if the database was created with an explicit encryption key, then Derby generates a verifyKey.dat file next to service.properties. The contents of verifyKey.dat are used at boot time to verify the encryption key before trying to decrypt the conglomerates.

          If you try to switch your encryption technique later on, Derby will fail to boot because it will not find the verification metadata needed by the other technique.

          2) If you change the boot password using boot attributes, this generates a new encryption key and re-encrypts the data.

          3) If you change the boot password using SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY, then a new encryption key is NOT generated and the data is NOT re-encrypted.

          4) Using SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY to change the boot password strikes me as a little weird. The usage, however, goes back a long way, at least as far back as Cloudscape 3.5. SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY was originally designed to store Derby properties in the property conglomerate. The boot password is not a property. It is a connection URL attribute. Furthermore, the associated, generated encryption key is not stored in the properties conglomerate. Instead, it is stored in service.properties. I don't think that we should be manipulating the contents of service.properties with a procedure which was designed to manage the property conglomerate. I support Knut's suggestion on DERBY-4328 that we should introduce a new SYSCS_UTIL.SYSCS_CHANGE_BOOT_PASSWORD procedure for changing the boot password on the fly. I would support deprecating the use of SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY to change the boot password; we could raise an error telling users to use SYSCS_UTIL.SYSCS_CHANGE_BOOT_PASSWORD instead. To mitigate the backward compatibility problems, we could phase in this change over the course of a couple releases, warning people in 10.10 that the old usage will be deprecated in 10.11.

          Show
          Rick Hillegas added a comment - Thanks for the repro and the additional analysis, Dag. Here are some more observations: 1) When you create an encrypted database, you specify either an explicit encryption key or a boot password. If you specify a boot password, then Derby generates an encryption key for you. You can't go back and forth between these techniques. Once you specify a boot password, you can't try to figure out the generated encryption key and boot the database using the encryption key. Similarly, if you create the database with an explicit encryption key, you can't try to introduce a boot password later on. This is because Derby stores different verification metadata for the two techniques. The verification metadata is used to catch bad boot passwords and encryption keys before actually trying to decrypt the conglomerates. If Derby can catch a bad boot password or encryption key before trying to decrypt the conglomerates, then Derby can raise an intelligible error message. If the database was created with a boot password, then Derby expects to find an encryptedBootPassword attribute in service.properties. The value of this attribute includes a digest used at boot time to verify the boot password before trying to decrypt the conglomerates. In contrast, if the database was created with an explicit encryption key, then Derby generates a verifyKey.dat file next to service.properties. The contents of verifyKey.dat are used at boot time to verify the encryption key before trying to decrypt the conglomerates. If you try to switch your encryption technique later on, Derby will fail to boot because it will not find the verification metadata needed by the other technique. 2) If you change the boot password using boot attributes, this generates a new encryption key and re-encrypts the data. 3) If you change the boot password using SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY, then a new encryption key is NOT generated and the data is NOT re-encrypted. 4) Using SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY to change the boot password strikes me as a little weird. The usage, however, goes back a long way, at least as far back as Cloudscape 3.5. SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY was originally designed to store Derby properties in the property conglomerate. The boot password is not a property. It is a connection URL attribute. Furthermore, the associated, generated encryption key is not stored in the properties conglomerate. Instead, it is stored in service.properties. I don't think that we should be manipulating the contents of service.properties with a procedure which was designed to manage the property conglomerate. I support Knut's suggestion on DERBY-4328 that we should introduce a new SYSCS_UTIL.SYSCS_CHANGE_BOOT_PASSWORD procedure for changing the boot password on the fly. I would support deprecating the use of SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY to change the boot password; we could raise an error telling users to use SYSCS_UTIL.SYSCS_CHANGE_BOOT_PASSWORD instead. To mitigate the backward compatibility problems, we could phase in this change over the course of a couple releases, warning people in 10.10 that the old usage will be deprecated in 10.11.
          Hide
          Dag H. Wanvik added a comment -

          I think the problem can only occur when changing the bootpassword with SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY. If one tries to change it at boot time, the boot would fail since S-bogus couldn't decrypt the data and the boot will fail with the same symptom as seen in DERBY-2687.
          Apparently, when using SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY, the data isn't being re-encrypted with a new secret key, and the old one gets lost
          as explained above.

          Show
          Dag H. Wanvik added a comment - I think the problem can only occur when changing the bootpassword with SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY. If one tries to change it at boot time, the boot would fail since S-bogus couldn't decrypt the data and the boot will fail with the same symptom as seen in DERBY-2687 . Apparently, when using SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY, the data isn't being re-encrypted with a new secret key, and the old one gets lost as explained above.
          Dag H. Wanvik made changes -
          Attachment derby-5622-instrumentation.diff [ 12531008 ]
          Attachment repro.sh [ 12531009 ]
          Hide
          Dag H. Wanvik added a comment -

          Uploading script + instrumentation patch which corrupts the db I believe. It can't be booted using any of the three involved boot passwords after we're done.

          Show
          Dag H. Wanvik added a comment - Uploading script + instrumentation patch which corrupts the db I believe. It can't be booted using any of the three involved boot passwords after we're done.
          Hide
          Dag H. Wanvik added a comment - - edited

          Given an encrypted secret key S, which is stored in service.properties after having been encrypted with the boot password BO, we have stored

          S' = encrypt(S, BO)

          The payload data D exists encrypted using S:

          D' = encrypt(D, S)

          so inverse, when we boot up again:

          Using BO we do

          S = decrypt(S', BO)

          check digest(S) against stored digest, and get at the data:

          D = decrypt(D', S).

          Now, when we try to change the boot password in the pathological case, we do this (using a bogus old boot password):

          S-bogus = decrypt(S', B-bogus)

          next, the check of S-bogus against digest and IV (initialization vector) would normally fail it, but it could happen that it was erroneously accepted. The what happen is we store using the new boot password (BN) onto service.properties:

          S-bogus' = encrypt(S-bogus, BN)

          Meanwhile, the data is still encrypted with S; we don't reencrypt everything after changing the boot password.

          Next time we boot up again we do:

          S-bogus = decrypt(S-bogus', BN)

          which works nicely and passes the digest since we use the correct new boot password, BN.

          Now, trying to get the data:

          decrypt(D', S-bogus)

          will yield garbage, since we'd need to use S to get the correct data out.

          So, to demonstrate this, I think you'll need to use a bogus old boot key, make it pass the digest and IV check using instrumentation of sorts, then shutdown, and reboot to see if your data makes sense. They shouldn't.

          Show
          Dag H. Wanvik added a comment - - edited Given an encrypted secret key S, which is stored in service.properties after having been encrypted with the boot password BO, we have stored S' = encrypt(S, BO) The payload data D exists encrypted using S: D' = encrypt(D, S) so inverse, when we boot up again: Using BO we do S = decrypt(S', BO) check digest(S) against stored digest, and get at the data: D = decrypt(D', S). Now, when we try to change the boot password in the pathological case, we do this (using a bogus old boot password): S-bogus = decrypt(S', B-bogus) next, the check of S-bogus against digest and IV (initialization vector) would normally fail it, but it could happen that it was erroneously accepted. The what happen is we store using the new boot password (BN) onto service.properties: S-bogus' = encrypt(S-bogus, BN) Meanwhile, the data is still encrypted with S; we don't reencrypt everything after changing the boot password. Next time we boot up again we do: S-bogus = decrypt(S-bogus', BN) which works nicely and passes the digest since we use the correct new boot password, BN. Now, trying to get the data: decrypt(D', S-bogus) will yield garbage, since we'd need to use S to get the correct data out. So, to demonstrate this, I think you'll need to use a bogus old boot key, make it pass the digest and IV check using instrumentation of sorts, then shutdown, and reboot to see if your data makes sense. They shouldn't.
          Hide
          Rick Hillegas added a comment -

          Thanks, Dag. The discussion on that issue suggests that there is a way to change the boot password so that the system becomes unbootable. That may be the data corruption case you indicate. My simple attempts to script that problem are not succeeding. I have tried commenting out the digest verification. That causes the ClassNotFoundException when I try to give a bad boot password, but I have not succeeded in creating an unbootable database.

          Can you help me understand how to script the data corruption? Instructions which describe how to fake a digest collision by instrumenting the engine are fine, I just need to be able to arrive at an unbootable database. Thanks.

          Show
          Rick Hillegas added a comment - Thanks, Dag. The discussion on that issue suggests that there is a way to change the boot password so that the system becomes unbootable. That may be the data corruption case you indicate. My simple attempts to script that problem are not succeeding. I have tried commenting out the digest verification. That causes the ClassNotFoundException when I try to give a bad boot password, but I have not succeeded in creating an unbootable database. Can you help me understand how to script the data corruption? Instructions which describe how to fake a digest collision by instrumenting the engine are fine, I just need to be able to arrive at an unbootable database. Thanks.
          Show
          Dag H. Wanvik added a comment - Please see discussion here: https://issues.apache.org/jira/browse/DERBY-2687?focusedCommentId=13209225&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13209225
          Hide
          Rick Hillegas added a comment -

          I am trying to figure out the data corruption entailed here. I can see a code path by which supplying a wrong boot password can stumble forward to die on a ClassNotFoundException. But after that, the correct boot password still boots the database. Can you help me understand how to trigger a data corruption? Thanks.

          Show
          Rick Hillegas added a comment - I am trying to figure out the data corruption entailed here. I can see a code path by which supplying a wrong boot password can stumble forward to die on a ClassNotFoundException. But after that, the correct boot password still boots the database. Can you help me understand how to trigger a data corruption? Thanks.
          Dag H. Wanvik made changes -
          Link This issue is related to DERBY-2687 [ DERBY-2687 ]
          Dag H. Wanvik made changes -
          Field Original Value New Value
          Description There are two issues, already seen in DERBY-2687:

             "the boot issue": there is a 1/2**16 chance that a wrong bootpassword will allow boot to proceed (but since its decoded key is wrong the boot will fail).
             "the oassword change" issue: similarly, there is a chance that the wrong bootpassword will be accepted trying to change it via
              SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY('bootPassword', ...) at least for algorithms that do not check IV (initialization vector) in addition to the
              digest, e.g. "DES/ECB/NoPadding"

          The latter case may lead to data corruption, cf. DERBY-2687 discussion. I think the risk is fairly low, though: One would need to have execution permission to change the property if SQL authorization is used, and in most scenarios the supplied existing password would be correct. But since the results can be bad, it would be good to reduce or eliminate the risk.
          There are two issues, already seen in DERBY-2687:

             "the boot issue": there is a 1/2**16 chance that a wrong bootPassword will allow boot to proceed (but since its decoded key is wrong the boot will fail).
             "the password change" issue: similarly, there is a chance that the wrong bootPassword will be accepted trying to change it via
              SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY('bootPassword', ...) at least for algorithms that do not check IV (initialization vector) in addition to the
              digest, e.g. "DES/ECB/NoPadding"

          The latter case may lead to data corruption, cf. DERBY-2687 discussion. I think the risk is fairly low, though: One would need to have execution permission to change the property if SQL authorization is used, and in most scenarios the supplied existing password would be correct. But since the results can be bad, it would be good to reduce or eliminate the risk.
          Dag H. Wanvik created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Dag H. Wanvik
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development