Derby
  1. Derby
  2. DERBY-5526

on upgrade from 10.5 to 10.8.2 , getting ERROR XBM0A: The database directory 'C:\cygwin\home\debugfat\clientdb' exists. However, it does not contain the expected 'service.properties' file.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.8.2.2
    • Fix Version/s: 10.8.3.0, 10.9.1.0
    • Component/s: Store
    • Labels:
      None
    • Issue & fix info:
      Patch Available
    • Bug behavior facts:
      Regression

      Description

      I have very few details yet, but user upgrading from 10.5 to 10.8.2 reports this error in one of their tests creating a database.

      Caused by: java.sql.SQLException: Failed to create database 'clientdb', see the next exception for details.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      ... 71 more
      Caused by: java.sql.SQLException: The database directory 'C:\cygwin\home\debugfat\clientdb' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
      ... 68 more
      Caused by: ERROR XBM0A: The database directory 'C:\cygwin\home\debugfat\clientdb' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again.
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.services.monitor.StorageFactoryService.vetService(Unknown Source)
      at org.apache.derby.impl.services.monitor.StorageFactoryService.access$600(Unknown Source)
      at org.apache.derby.impl.services.monitor.StorageFactoryService$9.run(Unknown Source)
      at java.security.AccessController.doPrivileged(AccessController.java:254)
      at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown Source)
      ... 68 more
      caused by: ERROR XBM0A: The database directory 'C:\cygwin\home\debugfat\clientdb' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again.
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.services.monitor.StorageFactoryService.vetService(Unknown Source)
      at org.apache.derby.impl.services.monitor.StorageFactoryService.access$600(Unknown Source)
      at org.apache.derby.impl.services.monitor.StorageFactoryService$9.run(Unknown Source)
      at java.security.AccessController.doPrivileged(AccessController.java:254)
      at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown Source)
      ... 68 more

      DERBY-4589 is where vetService was added, but I am not sure what has changed from 10.5. I don't see any release note associated with DERBY-4589. I will test an existing database scenario with 10.5 vs 10.8., vetService just checks that service.properties exists.

      Verify that the service directory looks ok before objecting that the database
      already exists.
      */
      private void vetService( StorageFactory storageFactory, String serviceName ) throws StandardException
      {
      // check for existence of service.properties descriptor file
      StorageFile service_properties = storageFactory.newStorageFile( PersistentService.PROPERTIES_NAME );

      if ( !service_properties.exists() )

      { throw StandardException.newException ( SQLState.SERVICE_PROPERTIES_MISSING, serviceName, PersistentService.PROPERTIES_NAME ); }

      }

      1. derby-5526_preliminary_diff.txt
        0.9 kB
        Kathey Marsden
      2. derby-5526_diff.txt
        3 kB
        Kathey Marsden

        Issue Links

          Activity

          Hide
          Kathey Marsden added a comment -

          The failure occurs in new code added in DERBY-4589, so linking to that issue.

          Show
          Kathey Marsden added a comment - The failure occurs in new code added in DERBY-4589 , so linking to that issue.
          Hide
          Kathey Marsden added a comment -

          Trying to create a database on an empty directory with 10.5 vs 10.8 I see the message and SQL State have changed. With 10.5 it is:
          Caused by: ERROR XBM0J: Directory C:\cygwin\home\kmarsden\repro\DERBY-5526\wombat already exists.
          at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:286)
          at org.apache.derby.impl.services.monitor.StorageFactoryService$9.run(StorageFactoryService.java:670)
          at java.security.AccessController.doPrivileged(AccessController.java:251)
          at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(StorageFactoryService.j
          )
          at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1798)
          at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(BaseMonitor.java:1060)
          at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Monitor.java:596)
          at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:2446)
          ... 7 more
          SQLState=XBM0JDirectory C:\cygwin\home\kmarsden\repro\DERBY-5526\wombat already exists.
          java.sql.SQLException: Directory C:\cygwin\home\kmarsden\repro\DERBY-5526\wombat already exists.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:95)
          at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:256)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:391
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:346)
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2204)
          at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:2452)
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:396)
          at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:68)
          at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:238)
          at org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:119)
          at java.sql.DriverManager.getConnection(DriverManager.java:322)
          at java.sql.DriverManager.getConnection(DriverManager.java:273)

          With 10.8 it is:

          Caused by: ERROR XBM0A: The database directory 'C:\cygwin\home\kmarsden\repro\DERBY-5526\wombat' exists. However, it doe
          s not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this data
          base. You may want to delete this directory and try creating the database again.
          at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:303)
          at org.apache.derby.impl.services.monitor.StorageFactoryService.vetService(StorageFactoryService.java:715)
          at org.apache.derby.impl.services.monitor.StorageFactoryService.access$600(StorageFactoryService.java:64)
          at org.apache.derby.impl.services.monitor.StorageFactoryService$9.run(StorageFactoryService.java:663)
          at java.security.AccessController.doPrivileged(AccessController.java:251)
          at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(StorageFactoryService.java:642
          )
          at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1740)
          at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(BaseMonitor.java:1002)
          at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Monitor.java:596)
          at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:2534)
          ... 7 more

          Show
          Kathey Marsden added a comment - Trying to create a database on an empty directory with 10.5 vs 10.8 I see the message and SQL State have changed. With 10.5 it is: Caused by: ERROR XBM0J: Directory C:\cygwin\home\kmarsden\repro\ DERBY-5526 \wombat already exists. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:286) at org.apache.derby.impl.services.monitor.StorageFactoryService$9.run(StorageFactoryService.java:670) at java.security.AccessController.doPrivileged(AccessController.java:251) at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(StorageFactoryService.j ) at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1798) at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(BaseMonitor.java:1060) at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Monitor.java:596) at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:2446) ... 7 more SQLState=XBM0JDirectory C:\cygwin\home\kmarsden\repro\ DERBY-5526 \wombat already exists. java.sql.SQLException: Directory C:\cygwin\home\kmarsden\repro\ DERBY-5526 \wombat already exists. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:95) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:256) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:391 at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:346) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2204) at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:2452) at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:396) at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:68) at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:238) at org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:119) at java.sql.DriverManager.getConnection(DriverManager.java:322) at java.sql.DriverManager.getConnection(DriverManager.java:273) With 10.8 it is: Caused by: ERROR XBM0A: The database directory 'C:\cygwin\home\kmarsden\repro\ DERBY-5526 \wombat' exists. However, it doe s not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this data base. You may want to delete this directory and try creating the database again. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:303) at org.apache.derby.impl.services.monitor.StorageFactoryService.vetService(StorageFactoryService.java:715) at org.apache.derby.impl.services.monitor.StorageFactoryService.access$600(StorageFactoryService.java:64) at org.apache.derby.impl.services.monitor.StorageFactoryService$9.run(StorageFactoryService.java:663) at java.security.AccessController.doPrivileged(AccessController.java:251) at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(StorageFactoryService.java:642 ) at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1740) at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(BaseMonitor.java:1002) at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Monitor.java:596) at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(EmbedConnection.java:2534) ... 7 more
          Hide
          Kathey Marsden added a comment -

          I am curious about 2 things:
          1) Why did the error SQLState change?

          2) Is this error too specific regard with suggesting the user delete the directory? Perhaps this is a directory that just happens to exist with some other non-derby contents and the application just wants to warn that the directory exists so the user can put their database elsewhere.

          Thanks

          Kathey

          Show
          Kathey Marsden added a comment - I am curious about 2 things: 1) Why did the error SQLState change? 2) Is this error too specific regard with suggesting the user delete the directory? Perhaps this is a directory that just happens to exist with some other non-derby contents and the application just wants to warn that the directory exists so the user can put their database elsewhere. Thanks Kathey
          Hide
          Kathey Marsden added a comment -

          Looking at the code where vetService() is called:

          else

          { vetService( storageFactory, name ); throw StandardException.newException(SQLState.SERVICE_DIRECTORY_EXISTS_ERROR, getDirectoryPath( name)); }

          and the issue comments for DERBY-4589, I think the intention of the new error was to just produce a more sensible message for a half created database, but to keep the old error message for directories not related to Derby. I am thinking it would make sense for vetService to check both that service.properties doesn't exist and seg0 does exists to determine if that is the state of affairs. I will give that a try on the empty directory case.

          Show
          Kathey Marsden added a comment - Looking at the code where vetService() is called: else { vetService( storageFactory, name ); throw StandardException.newException(SQLState.SERVICE_DIRECTORY_EXISTS_ERROR, getDirectoryPath( name)); } and the issue comments for DERBY-4589 , I think the intention of the new error was to just produce a more sensible message for a half created database, but to keep the old error message for directories not related to Derby. I am thinking it would make sense for vetService to check both that service.properties doesn't exist and seg0 does exists to determine if that is the state of affairs. I will give that a try on the empty directory case.
          Hide
          Kathey Marsden added a comment -

          Attaching a preliminary patch with my approach for this issue. Still need to add a test, run tests etc.

          The patch will try to better check that the database directory is really an aborted database creation and not just a user directory with other data by checking for the existence of the seg0 directory before throwing the new error about service.properties being missing.

          I verified with the empty directory we go back to just printing the error that the directory exists and that with the user database attached to DERBY-4589 we throw the XBMOA services.properties missing error.

          Show
          Kathey Marsden added a comment - Attaching a preliminary patch with my approach for this issue. Still need to add a test, run tests etc. The patch will try to better check that the database directory is really an aborted database creation and not just a user directory with other data by checking for the existence of the seg0 directory before throwing the new error about service.properties being missing. I verified with the empty directory we go back to just printing the error that the directory exists and that with the user database attached to DERBY-4589 we throw the XBMOA services.properties missing error.
          Hide
          Dag H. Wanvik added a comment -

          Looks like a good approach.
          Here is the order in which the database files are created (traced via limitAccessToOwner):

          $ java -Dderby.storage.useDefaultFilePermissions=false org.apache.derby.tools.ij
          ij version 10.9
          ij> connect 'jdbc:derby:foobat;create=true';
          limitAccessToOwner: derby.log
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/tmp
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/db.lck
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/dbex.lck
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/db.lck
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log/log.ctrl
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log/logmirror.ctrl
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log/log1.dat
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/seg0
          :
          limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/service.properties
          ij>

          So, the last file created is service.properties. I guess seg0 is a good (early) candidate, although it is not the first one to more or less safely say this is a Derby database directory.

          Show
          Dag H. Wanvik added a comment - Looks like a good approach. Here is the order in which the database files are created (traced via limitAccessToOwner): $ java -Dderby.storage.useDefaultFilePermissions=false org.apache.derby.tools.ij ij version 10.9 ij> connect 'jdbc:derby:foobat;create=true'; limitAccessToOwner: derby.log limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/tmp limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/db.lck limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/dbex.lck limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/db.lck limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log/log.ctrl limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log/logmirror.ctrl limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/log/log1.dat limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/seg0 : limitAccessToOwner: /export/home/dag/java/sb/sb1/foobat/service.properties ij> So, the last file created is service.properties. I guess seg0 is a good (early) candidate, although it is not the first one to more or less safely say this is a Derby database directory.
          Hide
          Mike Matrigali added a comment -

          checking for seg0 also seems fine to me, as if it does not exist there is really no hope of recovering any derby data. I would not check for "log" as it can be located on another disk and figuring that out requires access to the service.properties file.

          Show
          Mike Matrigali added a comment - checking for seg0 also seems fine to me, as if it does not exist there is really no hope of recovering any derby data. I would not check for "log" as it can be located on another disk and figuring that out requires access to the service.properties file.
          Hide
          Kathey Marsden added a comment -

          Here is the patch with the test added. There was already a test for the half created database, so added a fixture to test the regular case of just a user data directory.

          I think the seg0 check is best as tmp is too generic and as Mike says log can be offloaded elsewhere.

          I think the new message for service.properties could be improved to not so readily encourage deletion in the corrupt database case but did not change that with this patch.

          Show
          Kathey Marsden added a comment - Here is the patch with the test added. There was already a test for the half created database, so added a fixture to test the regular case of just a user data directory. I think the seg0 check is best as tmp is too generic and as Mike says log can be offloaded elsewhere. I think the new message for service.properties could be improved to not so readily encourage deletion in the corrupt database case but did not change that with this patch.
          Hide
          Dag H. Wanvik added a comment -

          Thanks, Kathey. I agree about the choice of seg0.

          Show
          Dag H. Wanvik added a comment - Thanks, Kathey. I agree about the choice of seg0.
          Hide
          Kathey Marsden added a comment -

          Resolving this issue. I checked in the fix to 10.9 and 10.8 branches.

          Show
          Kathey Marsden added a comment - Resolving this issue. I checked in the fix to 10.9 and 10.8 branches.
          Hide
          Knut Anders Hatlen added a comment -

          [bulk update: close all resolved issues that haven't had any activity the last year]

          Show
          Knut Anders Hatlen added a comment - [bulk update: close all resolved issues that haven't had any activity the last year]

            People

            • Assignee:
              Kathey Marsden
              Reporter:
              Kathey Marsden
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development