Derby
  1. Derby
  2. DERBY-1001

Rewrite 'store/encryptionKey.sql' to a JUnit test

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 10.3.1.4
    • Fix Version/s: 10.3.1.4
    • Component/s: Test
    • Labels:
      None

      Description

      This test has failed on Solaris10 for a long time, due to issues with the default security provider on this OS. See DERBY-788 for details.

      I consider rewriting this test as interresting because it allows us to see how things can be done in "the JUnit way".

      1) Run test with multiple encryption algorithms with minimal test code duplication.
      2) Special handling of exceptions for specific providers (PCKS11-Solaris).

      The rewritten test might cause some discussion on how we want to handle the issues mentioned above.

      1. derby-1001-3a-fixtestfailure.diff
        0.7 kB
        Kristian Waagan
      2. derby-1001-2a.stat
        0.8 kB
        Kristian Waagan
      3. derby-1001-2a.diff
        43 kB
        Kristian Waagan
      4. derby-1001-1a-preliminary.stat
        0.5 kB
        Kristian Waagan
      5. derby-1001-1a-preliminary.diff
        32 kB
        Kristian Waagan

        Issue Links

          Activity

          Hide
          Kristian Waagan added a comment -

          Closing issue. Problems running the test on other platforms/configurations should be filed in a separate Jira.

          Show
          Kristian Waagan added a comment - Closing issue. Problems running the test on other platforms/configurations should be filed in a separate Jira.
          Hide
          Kristian Waagan added a comment -

          Attached fix for the failing test. It is not quite clear to me if Derby actively needs read permissions for the directory to backup to, or if it is required as part of some other file system operation (e.g. create dir). Putting the backup in EXTINOUT instead of EXTOUT fixes the problem.

          I will investigate the issue a little more later, and I will also update the comments in the file accordingly (class JavaDoc) when the fix is verified. It also not clear to me why the test fails when running suites.All, but not individually or as part of store/_Suite.

          Committed 'derby-1001-3a-fixtestfailure.diff' to trunk with revision 536621.

          Show
          Kristian Waagan added a comment - Attached fix for the failing test. It is not quite clear to me if Derby actively needs read permissions for the directory to backup to, or if it is required as part of some other file system operation (e.g. create dir). Putting the backup in EXTINOUT instead of EXTOUT fixes the problem. I will investigate the issue a little more later, and I will also update the comments in the file accordingly (class JavaDoc) when the fix is verified. It also not clear to me why the test fails when running suites.All, but not individually or as part of store/_Suite. Committed 'derby-1001-3a-fixtestfailure.diff' to trunk with revision 536621.
          Hide
          Myrna van Lunteren added a comment -

          test fails in tinderbox:
          see: http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-536200.html

          org.apache.derbyTesting.functionTests.suites.All fail *************************************************************
          1) testBackupEncryptedDatabase(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyAESTest)java.sql.SQLException: The exception 'java.security.AccessControlException: access denied (java.io.FilePermission extout read)' was thrown while evaluating an expression.
          1) testBackupEncryptedDatabase(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyBlowfishTest)junit.framework.AssertionFailedError: Refusing to continue, database already exists <Database '/export/home/tmp/os136789/testingDerbyTinderBox/SunOS-5.10_i86pc-i386/org.apache.derbyTesting.functionTests.suites.All/extinout/encryptionKeyDBToBackup' not created, connection made to existing database instead.>
          2) testBackupEncryptedDatabase(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyDESTest)junit.framework.AssertionFailedError: Refusing to continue, database already exists <Database '/export/home/tmp/os136789/testingDerbyTinderBox/SunOS-5.10_i86pc-i386/org.apache.derbyTesting.functionTests.suites.All/extinout/encryptionKeyDBToBackup' not created, connection made to existing database instead.>

          Maybe the test needs to use singleUseDatabaseConfiguration ?

          Show
          Myrna van Lunteren added a comment - test fails in tinderbox: see: http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-536200.html org.apache.derbyTesting.functionTests.suites.All fail ************************************************************* 1) testBackupEncryptedDatabase(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyAESTest)java.sql.SQLException: The exception 'java.security.AccessControlException: access denied (java.io.FilePermission extout read)' was thrown while evaluating an expression. 1) testBackupEncryptedDatabase(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyBlowfishTest)junit.framework.AssertionFailedError: Refusing to continue, database already exists <Database '/export/home/tmp/os136789/testingDerbyTinderBox/SunOS-5.10_i86pc-i386/org.apache.derbyTesting.functionTests.suites.All/extinout/encryptionKeyDBToBackup' not created, connection made to existing database instead.> 2) testBackupEncryptedDatabase(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyDESTest)junit.framework.AssertionFailedError: Refusing to continue, database already exists <Database '/export/home/tmp/os136789/testingDerbyTinderBox/SunOS-5.10_i86pc-i386/org.apache.derbyTesting.functionTests.suites.All/extinout/encryptionKeyDBToBackup' not created, connection made to existing database instead.> Maybe the test needs to use singleUseDatabaseConfiguration ?
          Hide
          Kristian Waagan added a comment -

          Committed 'derby-1001-2a.diff' to trunk with revision 536156.

          Three different algorithms are currently tested: AES, DES and Blowfish.
          Streaming ciphers do not currently work with Derby.

          Show
          Kristian Waagan added a comment - Committed 'derby-1001-2a.diff' to trunk with revision 536156. Three different algorithms are currently tested: AES, DES and Blowfish. Streaming ciphers do not currently work with Derby.
          Hide
          Kristian Waagan added a comment -

          Attached 'derby-1001-2a.diff' for review/commit.

          I made the test rely on SupportFileSetup for cleaning up all the databases (deleting them), and added subclasses for three algorithms; AES, DES, Blowfish. I have no plans for adding more algorithms or variations atm. A run of store._Suite takes around 30 seconds on my machine.

          I'll commit the patch next week unless I get pushback.

          Show
          Kristian Waagan added a comment - Attached 'derby-1001-2a.diff' for review/commit. I made the test rely on SupportFileSetup for cleaning up all the databases (deleting them), and added subclasses for three algorithms; AES, DES, Blowfish. I have no plans for adding more algorithms or variations atm. A run of store._Suite takes around 30 seconds on my machine. I'll commit the patch next week unless I get pushback.
          Hide
          Kristian Waagan added a comment -

          Thanks Kathey and Sunitha for looking at the patch.
          It is not yet finished, but I will upload another version shortly.

          Regarding the comments:

          • I considered deleting the database directories, but couldn't find a utility to do so. There is one in the org...junit package, but it is package private. I'll look into adding the functionality, or maybe move the "deleteDirectory"-method to a public utility class.
          • Looking at the EncryptionSuite, I see we currently test three different algorithms. I'll add a test for each (DES, AES and Blowfish). I'm not sure if it is worth testing all the different variations (feedback mode, padding). Any opinions on this in the community?
          • We should test a streaming cipher (RC4/ARCFOUR), but Derby does not currently handle this (see DERBY-2552).
          • That the suites-methods are different is a bug. I'll fix them in the next patch.

          regards,

          Show
          Kristian Waagan added a comment - Thanks Kathey and Sunitha for looking at the patch. It is not yet finished, but I will upload another version shortly. Regarding the comments: I considered deleting the database directories, but couldn't find a utility to do so. There is one in the org...junit package, but it is package private. I'll look into adding the functionality, or maybe move the "deleteDirectory"-method to a public utility class. Looking at the EncryptionSuite, I see we currently test three different algorithms. I'll add a test for each (DES, AES and Blowfish). I'm not sure if it is worth testing all the different variations (feedback mode, padding). Any opinions on this in the community? We should test a streaming cipher (RC4/ARCFOUR), but Derby does not currently handle this (see DERBY-2552 ). That the suites-methods are different is a bug. I'll fix them in the next patch. regards,
          Hide
          Sunitha Kambhampati added a comment -

          Each of the tests- EncryptionKeyDESTest and the EncryptionKeyAESTest require a clean testrun directory..

          I ran the EncryptionKeyDESTest and then cleaned up my testrun directory and then ran the AES test and it worked OK in Eclipse as well as on command prompt. I was using 1.4.2 Sun JVM.

          kristian> 2) How many encryption algorithms/variations should we test?
          How about the algorithms that we test in EncryptionSuite. It would be interesting to see how much time the tests would take in this case..

          I just took a quick look at the suite() methods in the DES and AES test and was wondering why they were different.. In one, there is the SupportFilesSetup decorator but in AES test that is not used..

          Thanks.

          Show
          Sunitha Kambhampati added a comment - Each of the tests- EncryptionKeyDESTest and the EncryptionKeyAESTest require a clean testrun directory.. I ran the EncryptionKeyDESTest and then cleaned up my testrun directory and then ran the AES test and it worked OK in Eclipse as well as on command prompt. I was using 1.4.2 Sun JVM. kristian> 2) How many encryption algorithms/variations should we test? How about the algorithms that we test in EncryptionSuite. It would be interesting to see how much time the tests would take in this case.. I just took a quick look at the suite() methods in the DES and AES test and was wondering why they were different.. In one, there is the SupportFilesSetup decorator but in AES test that is not used.. Thanks.
          Hide
          Kristian Waagan added a comment -

          Nothing special should be required to run the test, but you do need to have the necessary encryption software. With a recent JVM you should be good to go, but I have only tested with Sun JVMs.
          What JVM did you run with?

          I was able to run the test(s) on Windows XP, both under Cygwin and at the command prompt (cmd.exe). I used the Java SE 6 platform.
          I know nothing about Eclipse, so I can't say whether running in it would make any difference.

          Show
          Kristian Waagan added a comment - Nothing special should be required to run the test, but you do need to have the necessary encryption software. With a recent JVM you should be good to go, but I have only tested with Sun JVMs. What JVM did you run with? I was able to run the test(s) on Windows XP, both under Cygwin and at the command prompt (cmd.exe). I used the Java SE 6 platform. I know nothing about Eclipse, so I can't say whether running in it would make any difference.
          Hide
          Kathey Marsden added a comment -

          I am having trouble running these tests on Windows XP using eclipse I get errors starting the database and also
          java.sql.SQLException: Startup failed due to missing functionality for org.apache.derby.iapi.services.crypto.CipherFactoryBuilder. Please ensure your classpath includes the correct Derby software.
          at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:305)
          at org.apache.derby.iapi.services.monitor.Monitor.missingImplementation(Monitor.java:656)
          at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:283)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:546)
          at org.apache.derby.iapi.services.monitor.Monitor.startSystemModule(Monitor.java:366)
          at org.apache.derby.impl.store.raw.RawStore.setupEncryptionEngines(RawStore.java:1183)
          at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:204)
          at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1994)
          at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:546)
          at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:419)
          at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:985)
          at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1994)
          at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:546)
          at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:419)
          at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:753)
          at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:188)
          at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1994)
          at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291)
          at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1829)
          at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(BaseMonitor.java:1017)
          at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Monitor.java:588)
          at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:1750)
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:281)
          at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73)
          at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:74)
          at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:209)
          at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:479)
          at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:423)
          at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.getConnection(EncryptionKeyTest.java:564)
          at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.getConnection(EncryptionKeyTest.java:476)
          at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.testCreateWithOddEncryptionKeyLength(EncryptionKeyTest.java:385)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:60)
          at java.lang.reflect.Method.invoke(Method.java:391)
          at junit.framework.TestCase.runTest(TestCase.java:154)
          at junit.framework.TestCase.runBare(TestCase.java:127)
          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:88)
          at junit.framework.TestResult$1.protect(TestResult.java:106)
          at junit.framework.TestResult.runProtected(TestResult.java:124)
          at junit.framework.TestResult.run(TestResult.java:109)
          at junit.framework.TestCase.run(TestCase.java:118)
          at junit.framework.TestSuite.runTest(TestSuite.java:208)
          at junit.framework.TestSuite.run(TestSuite.java:203)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.framework.TestResult.runProtected(TestResult.java:124)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:128)
          at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
          at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
          at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
          at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
          at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)

          Is there some special setup I need to do to run these tests?

          Thanks

          Kathey

          Show
          Kathey Marsden added a comment - I am having trouble running these tests on Windows XP using eclipse I get errors starting the database and also java.sql.SQLException: Startup failed due to missing functionality for org.apache.derby.iapi.services.crypto.CipherFactoryBuilder. Please ensure your classpath includes the correct Derby software. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:305) at org.apache.derby.iapi.services.monitor.Monitor.missingImplementation(Monitor.java:656) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:283) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:546) at org.apache.derby.iapi.services.monitor.Monitor.startSystemModule(Monitor.java:366) at org.apache.derby.impl.store.raw.RawStore.setupEncryptionEngines(RawStore.java:1183) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:204) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1994) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:546) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:419) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:985) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1994) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:546) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:419) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:753) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:188) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1994) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1829) at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(BaseMonitor.java:1017) at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Monitor.java:588) at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:1750) at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:281) at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73) at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:74) at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:209) at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:479) at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:423) at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.getConnection(EncryptionKeyTest.java:564) at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.getConnection(EncryptionKeyTest.java:476) at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.testCreateWithOddEncryptionKeyLength(EncryptionKeyTest.java:385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:60) at java.lang.reflect.Method.invoke(Method.java:391) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:88) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:128) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196) Is there some special setup I need to do to run these tests? Thanks Kathey
          Hide
          Kristian Waagan added a comment -

          Forgot to ask if there are any important test cases I have missed...

          Show
          Kristian Waagan added a comment - Forgot to ask if there are any important test cases I have missed...
          Hide
          Kristian Waagan added a comment -

          I have attached a preliminary patch for the JUnit test ('derby-1001-1a-preliminary.diff').
          Review is appreciated, and I also have the following questions/observations:

          1) I'm considering ditching the subclassing approach and instead create multiple suites in EncryptionKeyTest.suite for the various encryption algorithms.
          2) How many encryption algorithms/variations should we test?
          (currently one run takes around 12 seconds on my machine)
          3) Is the getLastSQLException-approch acceptable?
          4) Error reporting still seems to be kind of broken. Exceptions from the security provider do not show in JUnit/general, but they do appear in derby.log.
          5) I have implemented a solution for the security manager issues that is not in line with my own reasoning... (see previous comment). I hope to get away with creating two separate Jiras; one to add the doPrivileged-blocks in Derby, and one to update the test.

          I also need to remove the old test, and to wire the new one into a suite. These changes and feedback will be incorporated into the second patch that supersedes 1a.

          Show
          Kristian Waagan added a comment - I have attached a preliminary patch for the JUnit test ('derby-1001-1a-preliminary.diff'). Review is appreciated, and I also have the following questions/observations: 1) I'm considering ditching the subclassing approach and instead create multiple suites in EncryptionKeyTest.suite for the various encryption algorithms. 2) How many encryption algorithms/variations should we test? (currently one run takes around 12 seconds on my machine) 3) Is the getLastSQLException-approch acceptable? 4) Error reporting still seems to be kind of broken. Exceptions from the security provider do not show in JUnit/general, but they do appear in derby.log. 5) I have implemented a solution for the security manager issues that is not in line with my own reasoning... (see previous comment). I hope to get away with creating two separate Jiras; one to add the doPrivileged-blocks in Derby, and one to update the test. I also need to remove the old test, and to wire the new one into a suite. These changes and feedback will be incorporated into the second patch that supersedes 1a.
          Hide
          Kristian Waagan added a comment -

          I might be confused, so I'll write out my thoughts and people can
          correct me.

          Since the path we need (read) access to is specified by the user in the
          connection string, I thought these permissions should not be granted to
          the Derby codebase, but rather for the application codebase using Derby.
          This would mean the application would have to do a doPrivileged when
          restoring/creating the database. [1] says: "... the call to doPrivileged
          should be made in the code that wants to enable its privileges." After
          the connection has been made, Derby's "core permissions" should be
          sufficient to ensure normal operation again.

          However, when running the network server, the approach mentioned above
          will not work, and permissions to the backup locations would have to be
          granted to derby.jar, not the application code. Then adding
          doPrivileged-blocks to the derby.jar codebase is reasonable.

          I think also I forgot that our policy file is just an example and for a
          specific use (for instance running the tests). For a concrete
          deployment, the user/administrator must add the proper permissions. The
          fact that Derby was able to write the backup, but not read it, must be
          attributed to more extensive usage of doPrivileged-blocks in one code
          path over the other. I suppose the default policy file and the tests
          must be written to be compatible with each other, and I think they will
          be if we add more doPrivileged-blocks.

          My initial concern was why other actions in the area are wrapped in
          doPrivileged-blocks, while the backupRoot.exists() is not. Thought there
          could be was a reason for that.

          Does this make any sense?

          [1] http://java.sun.com/j2se/1.5.0/docs/guide/security/doprivileged.html

          Show
          Kristian Waagan added a comment - I might be confused, so I'll write out my thoughts and people can correct me. Since the path we need (read) access to is specified by the user in the connection string, I thought these permissions should not be granted to the Derby codebase, but rather for the application codebase using Derby. This would mean the application would have to do a doPrivileged when restoring/creating the database. [1] says: "... the call to doPrivileged should be made in the code that wants to enable its privileges." After the connection has been made, Derby's "core permissions" should be sufficient to ensure normal operation again. However, when running the network server, the approach mentioned above will not work, and permissions to the backup locations would have to be granted to derby.jar, not the application code. Then adding doPrivileged-blocks to the derby.jar codebase is reasonable. I think also I forgot that our policy file is just an example and for a specific use (for instance running the tests). For a concrete deployment, the user/administrator must add the proper permissions. The fact that Derby was able to write the backup, but not read it, must be attributed to more extensive usage of doPrivileged-blocks in one code path over the other. I suppose the default policy file and the tests must be written to be compatible with each other, and I think they will be if we add more doPrivileged-blocks. My initial concern was why other actions in the area are wrapped in doPrivileged-blocks, while the backupRoot.exists() is not. Thought there could be was a reason for that. Does this make any sense? [1] http://java.sun.com/j2se/1.5.0/docs/guide/security/doprivileged.html
          Hide
          Daniel John Debrunner added a comment -

          I would say b) is the correct solution. Any call in derby that could result in a security manager check/exception needs to be in a privileged block.

          However, I don't understand your concern to b) when you say "since the files Derby needs to read may be anywhere".

          Could you explain more? Thanks.

          Show
          Daniel John Debrunner added a comment - I would say b) is the correct solution. Any call in derby that could result in a security manager check/exception needs to be in a privileged block. However, I don't understand your concern to b) when you say "since the files Derby needs to read may be anywhere". Could you explain more? Thanks.
          Hide
          Kristian Waagan added a comment -

          I have rewritten the test to JUnit, but have problems with the file permissions.
          As far as I can see, the offending call is made from JUnit, propagates through derby.jar and fails with this stack trace:
          1) testRestoreFrom(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyDESTest)java.security.AccessControlException: access denied (java.io.FilePermission extinout/encryptionKeyDBToRestoreFrom read)
          at java.security.AccessControlContext.checkPermission(AccessControlContext.java:264)
          at java.security.AccessController.checkPermission(AccessController.java:427)
          at java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
          at java.lang.SecurityManager.checkRead(SecurityManager.java:871)
          at java.io.File.exists(File.java:700)
          at org.apache.derby.impl.services.monitor.StorageFactoryService.recreateServiceRoot(StorageFactoryService.java:579)
          at org.apache.derby.impl.services.monitor.StorageFactoryService.getServiceProperties(StorageFactoryService.java:252)
          at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1544)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:991)
          at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:542)
          at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:1632)
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:223)
          at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73)
          at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:74)
          at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:210)
          at org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:117)
          at java.sql.DriverManager.getConnection(DriverManager.java:525)
          at java.sql.DriverManager.getConnection(DriverManager.java:171)
          at org.apache.derbyTesting.junit.TestConfiguration.getConnection(TestConfiguration.java:456)
          at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:236)
          at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.testRestoreFrom(EncryptionKeyTest.java:166)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76)

          The exception is thrown both when running as JUnit standalone and through the test harness.

          What is the correct solution to this problem?
          a) Add read permission to all for $

          {user.dir}

          $

          {/}extinout${/}

          -
          Added to all to include junit.jar, since it can be anywhere on the system.
          b) Wrap relevant file operations derby.jar in doPrivileged.
          c) Would it be possible to wrap "something" inside a doPrivileged in derbyTesting.jar
          d) Other solutions?

          I think maybe b) is not the way to go, since the files Derby needs to read may be anywhere.
          I lean towards considering this problem as a pure test problem, but I might have misunderstood.

          Show
          Kristian Waagan added a comment - I have rewritten the test to JUnit, but have problems with the file permissions. As far as I can see, the offending call is made from JUnit, propagates through derby.jar and fails with this stack trace: 1) testRestoreFrom(org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyDESTest)java.security.AccessControlException: access denied (java.io.FilePermission extinout/encryptionKeyDBToRestoreFrom read) at java.security.AccessControlContext.checkPermission(AccessControlContext.java:264) at java.security.AccessController.checkPermission(AccessController.java:427) at java.lang.SecurityManager.checkPermission(SecurityManager.java:532) at java.lang.SecurityManager.checkRead(SecurityManager.java:871) at java.io.File.exists(File.java:700) at org.apache.derby.impl.services.monitor.StorageFactoryService.recreateServiceRoot(StorageFactoryService.java:579) at org.apache.derby.impl.services.monitor.StorageFactoryService.getServiceProperties(StorageFactoryService.java:252) at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1544) at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:991) at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:542) at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:1632) at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:223) at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73) at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:74) at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:210) at org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:117) at java.sql.DriverManager.getConnection(DriverManager.java:525) at java.sql.DriverManager.getConnection(DriverManager.java:171) at org.apache.derbyTesting.junit.TestConfiguration.getConnection(TestConfiguration.java:456) at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:236) at org.apache.derbyTesting.functionTests.tests.store.EncryptionKeyTest.testRestoreFrom(EncryptionKeyTest.java:166) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76) The exception is thrown both when running as JUnit standalone and through the test harness. What is the correct solution to this problem? a) Add read permission to all for $ {user.dir} $ {/}extinout${/} - Added to all to include junit.jar, since it can be anywhere on the system. b) Wrap relevant file operations derby.jar in doPrivileged. c) Would it be possible to wrap "something" inside a doPrivileged in derbyTesting.jar d) Other solutions? I think maybe b) is not the way to go, since the files Derby needs to read may be anywhere. I lean towards considering this problem as a pure test problem, but I might have misunderstood.

            People

            • Assignee:
              Kristian Waagan
              Reporter:
              Kristian Waagan
            • Votes:
              1 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development