Derby
  1. Derby
  2. DERBY-5916

java.lang.NullPointerException org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop() connecting to network server

    Details

    • Urgency:
      Normal
    • Bug behavior facts:
      Seen in production

      Description

      I got a report of the exception below trying to connect to database with absolute path and network server and the database name attribute. I haven't gotten information on the derby version or platform, Below is the url I received with some characters replaced.

      ij> connect 'jdbc:derby://localhost:1527/;databaseName=/home/uxxxx/Installs/hxxx_ext/mxxxxxxxx_db;create=true' ;

      java.lang.NullPointerException
      at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.stop(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.shutdown(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
      at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
      at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
      at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
      at org.apache.derby.impl.drda.Database.makeConnection(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
      at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
      Cleanup action completed
      2012-08-31 22:41:09.216 GMT Thread[DRDAConnThread_10,5,main] (DATABASE = ), (DRDAID =

      {1}

      ), Java exception: ': java.lang.NullPointerException'.

      I wanted to go ahead and file it even without much information as I noticed there was a similar issue reported on the list but never filed:
      http://old.nabble.com/Random-DRDA-Error-on-IBM-J9-JVM-td33532717.html#a33532717

        Activity

        Hide
        Knut Anders Hatlen added a comment -

        [bulk update] Close all resolved issues that haven't been updated for more than one year.

        Show
        Knut Anders Hatlen added a comment - [bulk update] Close all resolved issues that haven't been updated for more than one year.
        Hide
        Mamta A. Satoor added a comment -

        Backported to 10.5 with revision 1395788

        Show
        Mamta A. Satoor added a comment - Backported to 10.5 with revision 1395788
        Hide
        Mamta A. Satoor added a comment -

        Backported to 10.6 with revision 1395712

        Show
        Mamta A. Satoor added a comment - Backported to 10.6 with revision 1395712
        Hide
        Mamta A. Satoor added a comment -

        Backported to 10.7 with revision 1395709

        Show
        Mamta A. Satoor added a comment - Backported to 10.7 with revision 1395709
        Hide
        Mamta A. Satoor added a comment -

        Backported to 10.8 with revision 1395186

        Show
        Mamta A. Satoor added a comment - Backported to 10.8 with revision 1395186
        Hide
        Mamta A. Satoor added a comment -

        Backported to 10.9 with revision 1395148

        Show
        Mamta A. Satoor added a comment - Backported to 10.9 with revision 1395148
        Hide
        Mamta A. Satoor added a comment -

        Committed changes into trunk with revision 1394883 with following commit comments
        ********************
        DERBY-5916 (java.lang.NullPointerException org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop() connecting to network server)

        Check for the nullability of storageFactory before using it in shutdown code
        ********************

        Show
        Mamta A. Satoor added a comment - Committed changes into trunk with revision 1394883 with following commit comments ******************** DERBY-5916 (java.lang.NullPointerException org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop() connecting to network server) Check for the nullability of storageFactory before using it in shutdown code ********************
        Hide
        Mamta A. Satoor added a comment -

        BTW, just for testing purposes, I set storageFactory to null at the beginning of BaseDataFileFactory.stop() and verified that we don't run into NPE after adding storageFactory nullability check in BaseDataFileFactory.stop() and in BaseDataFileFactory.removeStubs()

        Show
        Mamta A. Satoor added a comment - BTW, just for testing purposes, I set storageFactory to null at the beginning of BaseDataFileFactory.stop() and verified that we don't run into NPE after adding storageFactory nullability check in BaseDataFileFactory.stop() and in BaseDataFileFactory.removeStubs()
        Hide
        Bryan Pendleton added a comment -

        > in addition to nullabiliuty check before calling storageFactory,shutdown(), I will also add a nullability check inside removeStubs().

        +1. This seems like a simple and solid approach.

        Show
        Bryan Pendleton added a comment - > in addition to nullabiliuty check before calling storageFactory,shutdown(), I will also add a nullability check inside removeStubs(). +1. This seems like a simple and solid approach.
        Hide
        Mamta A. Satoor added a comment -

        I want to clarify that removeTempDirectory() call from BaseDataFileFactory.stop() will not result into a NPE even if storageFactory is null. This is because removeTempDirectory() checks for the nullability of storageFactory. Part of code in BaseDataFileFactory.stop() is as follows
        removeTempDirectory();

        if (isReadOnly()) // do enough to close all files, then return

        { storageFactory.shutdown(); return; }

        // re-enable stub removal until a better method can be found.
        // only remove stub if caches are cleaned
        if (removeStubsOK && OK)
        removeStubs();

        With this existing code in trunk, we can run into NPE if storageFactory is null and isReadOnly() returns true. If isReadOnly() returns false, and next if condition is true, then removeStubs() can run into NPE because we do not check for storageFactory nullability.

        So, in addition to nullabiliuty check before calling storageFactory,shutdown(), I will also add a nullability check inside removeStubs().

        Show
        Mamta A. Satoor added a comment - I want to clarify that removeTempDirectory() call from BaseDataFileFactory.stop() will not result into a NPE even if storageFactory is null. This is because removeTempDirectory() checks for the nullability of storageFactory. Part of code in BaseDataFileFactory.stop() is as follows removeTempDirectory(); if (isReadOnly()) // do enough to close all files, then return { storageFactory.shutdown(); return; } // re-enable stub removal until a better method can be found. // only remove stub if caches are cleaned if (removeStubsOK && OK) removeStubs(); With this existing code in trunk, we can run into NPE if storageFactory is null and isReadOnly() returns true. If isReadOnly() returns false, and next if condition is true, then removeStubs() can run into NPE because we do not check for storageFactory nullability. So, in addition to nullabiliuty check before calling storageFactory,shutdown(), I will also add a nullability check inside removeStubs().
        Hide
        Mamta A. Satoor added a comment -

        It definitely is a good idea to check if storageFactory is null before calling shutdown on it in BaseDataFileFactory.stop(). But I believe we may have gotten NPE somewhere else, especially if we are talking about Derby 10.6 or prior.

        I spent some more time looking at the code in trunk and right before the following check in BaseDataFileFactory.stop()
        if (isReadOnly()) // do enough to close all files, then return

        { storageFactory.shutdown(); return; }

        there is following code
        removeTempDirectory();
        The call to removeTempDirectory() leads to
        public final Object run() throws IOException, StandardException
        {
        switch( actionCode)
        {
        .....;

        case REMOVE_TEMP_DIRECTORY_ACTION:
        StorageFile tempDir = storageFactory.getTempDir();
        if( tempDir != null)
        tempDir.deleteAll();
        return null;
        If storageFactory was null, we would have gotten NPE here before we even reach the code where we call storageFactory.stop() in BaseDataFileFactory.stop()

        I will be interested in knowing what version of Derby was being used by the user of this jira. If my theory is correct, then may be they were using 10.6 or earlier release. In 10.6 and prior codelines, we have following piece of code in BaseDataFileFactory.stop() towards the beginning of the method
        long shutdownTime = System.currentTimeMillis();
        boolean logBootTrace = PropertyUtil.getSystemBoolean(Property.LOG_BOOT_TRACE);
        istream.println(LINE);
        logMsg("\n" + new Date() +
        Note, that we call istream.println(LINE) without checking if istream is null. This was fixed by Knut as part of DERBY-4873 but that fix went into 10.7. The fix was to use the helper method logMsg() which looks for istream being null and initializing it properly in case it is found null. I think it will be good idea to backport this change from DERBY-4873 to 10.6 and earlier releases.

        Show
        Mamta A. Satoor added a comment - It definitely is a good idea to check if storageFactory is null before calling shutdown on it in BaseDataFileFactory.stop(). But I believe we may have gotten NPE somewhere else, especially if we are talking about Derby 10.6 or prior. I spent some more time looking at the code in trunk and right before the following check in BaseDataFileFactory.stop() if (isReadOnly()) // do enough to close all files, then return { storageFactory.shutdown(); return; } there is following code removeTempDirectory(); The call to removeTempDirectory() leads to public final Object run() throws IOException, StandardException { switch( actionCode) { .....; case REMOVE_TEMP_DIRECTORY_ACTION: StorageFile tempDir = storageFactory.getTempDir(); if( tempDir != null) tempDir.deleteAll(); return null; If storageFactory was null, we would have gotten NPE here before we even reach the code where we call storageFactory.stop() in BaseDataFileFactory.stop() I will be interested in knowing what version of Derby was being used by the user of this jira. If my theory is correct, then may be they were using 10.6 or earlier release. In 10.6 and prior codelines, we have following piece of code in BaseDataFileFactory.stop() towards the beginning of the method long shutdownTime = System.currentTimeMillis(); boolean logBootTrace = PropertyUtil.getSystemBoolean(Property.LOG_BOOT_TRACE); istream.println(LINE); logMsg("\n" + new Date() + Note, that we call istream.println(LINE) without checking if istream is null. This was fixed by Knut as part of DERBY-4873 but that fix went into 10.7. The fix was to use the helper method logMsg() which looks for istream being null and initializing it properly in case it is found null. I think it will be good idea to backport this change from DERBY-4873 to 10.6 and earlier releases.
        Hide
        Mamta A. Satoor added a comment -

        Kathey, do we know what release of Derby was getting used?

        Show
        Mamta A. Satoor added a comment - Kathey, do we know what release of Derby was getting used?
        Hide
        Kathey Marsden added a comment -

        No Sorry Mamta. I wasn't able to get the derby.log.

        Show
        Kathey Marsden added a comment - No Sorry Mamta. I wasn't able to get the derby.log.
        Hide
        Mamta A. Satoor added a comment -

        Attaching a patch with simple change to look if storageFactory is null before calling shutdown on it in BaseDataFileFactory.stop(). This should not break anything but I am running the tests just in case.

        Show
        Mamta A. Satoor added a comment - Attaching a patch with simple change to look if storageFactory is null before calling shutdown on it in BaseDataFileFactory.stop(). This should not break anything but I am running the tests just in case.
        Hide
        Mamta A. Satoor added a comment -

        Hi Kathey, do you happen to have the derby.log for the above NPE case?

        Show
        Mamta A. Satoor added a comment - Hi Kathey, do you happen to have the derby.log for the above NPE case?
        Hide
        Mike Matrigali added a comment -

        triaged for 10.10

        Without a repro, probably not much can be done. We should at least implement the subtask
        DERBY-5922, and maybe that will provide information in the field why this is happening.

        Show
        Mike Matrigali added a comment - triaged for 10.10 Without a repro, probably not much can be done. We should at least implement the subtask DERBY-5922 , and maybe that will provide information in the field why this is happening.
        Hide
        Bryan Pendleton added a comment -

        This part of the stack is interesting:

        at org.apache.derby.impl.services.monitor.TopService.shutdown(Unknown Source)
        at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)

        There's only one place where bootService calls shutdown like this, and it's
        in the catch (Throwable) block.

        It sounds like there was such a severe error trying to start up the database that
        we were trying to abort the startup and shutdown the parts that did start.

        But we were in a state where, having failed to start up, we couldn't shut down, either.

        BaseDataFileFactory.stop is very thorough with its null pointer checks, so it's not
        easy to see how that code could get a NPE.

        But I notice that line 532, the call to storageFactory.shutdown in this:

        if (isReadOnly()) // do enough to close all files, then return

        { storageFactory.shutdown(); return; }

        isn't protected by "if( storageFactory != null )"

        Perhaps you could reproduce this artificially by arranging by forcing
        the startup code to throw an exception at just the right point...

        Show
        Bryan Pendleton added a comment - This part of the stack is interesting: at org.apache.derby.impl.services.monitor.TopService.shutdown(Unknown Source) at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source) There's only one place where bootService calls shutdown like this, and it's in the catch (Throwable) block. It sounds like there was such a severe error trying to start up the database that we were trying to abort the startup and shutdown the parts that did start. But we were in a state where, having failed to start up, we couldn't shut down, either. BaseDataFileFactory.stop is very thorough with its null pointer checks, so it's not easy to see how that code could get a NPE. But I notice that line 532, the call to storageFactory.shutdown in this: if (isReadOnly()) // do enough to close all files, then return { storageFactory.shutdown(); return; } isn't protected by "if( storageFactory != null )" Perhaps you could reproduce this artificially by arranging by forcing the startup code to throw an exception at just the right point...

          People

          • Assignee:
            Mamta A. Satoor
            Reporter:
            Kathey Marsden
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development