Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1959

Better error message for missing namenode directory

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Starting the namenode with a missing NN directory currently results in two stack traces, "Expecting a line not the end of stream" from DF and an NPE. Let's make this more user-friendly.

        Activity

        Eli Collins created issue -
        Hide
        Eli Collins added a comment -

        Patch attached.

        Show
        Eli Collins added a comment - Patch attached.
        Eli Collins made changes -
        Field Original Value New Value
        Attachment hdfs-1959-1.patch [ 12479699 ]
        Eli Collins made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Todd Lipcon added a comment -

        +1 pending hudson results

        Show
        Todd Lipcon added a comment - +1 pending hudson results
        Hide
        Aaron T. Myers added a comment -

        I'm no committer, but that +1 is ridiculous. Here's my reasoning:

        1. The if (dir != null) check will prevent an NPE at that point, but you'll likely just get it somewhere else later that assumes that dir is non-null.
        2. The change in NameNodeResourceChecker is a fine one to make, but it doesn't address the root problem. It's now the place we're relying on to report this error condition. Thing is, the NameNodeResourceChecker checks can be entirely disabled via config, in which case this code will never be hit.

        How about we add a proper check for this condition somewhere early-on in Name Node initialization?

        Show
        Aaron T. Myers added a comment - I'm no committer, but that +1 is ridiculous. Here's my reasoning: The if (dir != null) check will prevent an NPE at that point, but you'll likely just get it somewhere else later that assumes that dir is non-null. The change in NameNodeResourceChecker is a fine one to make, but it doesn't address the root problem. It's now the place we're relying on to report this error condition. Thing is, the NameNodeResourceChecker checks can be entirely disabled via config, in which case this code will never be hit. How about we add a proper check for this condition somewhere early-on in Name Node initialization?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12479699/hdfs-1959-1.patch
        against trunk revision 1124459.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
        org.apache.hadoop.hdfs.TestFileConcurrentReader
        org.apache.hadoop.tools.TestJMXGet

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/574//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/574//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/574//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479699/hdfs-1959-1.patch against trunk revision 1124459. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.tools.TestJMXGet +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/574//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/574//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/574//console This message is automatically generated.
        Todd Lipcon made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Aaron T. Myers added a comment -

        On a second read, my #2 point is invalid. The call to nnResourceChecker.hasAvailableDiskSpace() will definitely happen on FSNamesystem initialization, regardless of config settings. Given this, the patch does indeed address the issue. I humbly withdraw my objections.

        Show
        Aaron T. Myers added a comment - On a second read, my #2 point is invalid. The call to nnResourceChecker.hasAvailableDiskSpace() will definitely happen on FSNamesystem initialization, regardless of config settings. Given this, the patch does indeed address the issue. I humbly withdraw my objections.
        Eli Collins made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags [Reviewed]
        Hide
        Eli Collins added a comment -

        Test failures are unrelated. I've committed this.

        Thanks Todd and atm!

        Show
        Eli Collins added a comment - Test failures are unrelated. I've committed this. Thanks Todd and atm!
        Eli Collins made changes -
        Resolution Fixed [ 1 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #678 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/678/)
        HDFS-1959. Better error message for missing namenode directory. Contributed by Eli Collins

        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1125879
        Files :

        • /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java
        • /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/hdfs/trunk/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #678 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/678/ ) HDFS-1959 . Better error message for missing namenode directory. Contributed by Eli Collins eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1125879 Files : /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeResourceChecker.java /hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/hdfs/trunk/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #675 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/675/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #675 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/675/ )
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        1d 3h 7m 1 Todd Lipcon 20/May/11 04:33
        Open Open Patch Available Patch Available
        2d 4h 44m 2 Eli Collins 22/May/11 09:16
        Patch Available Patch Available Resolved Resolved
        22s 1 Eli Collins 22/May/11 09:16
        Resolved Resolved Closed Closed
        176d 16h 36m 1 Arun C Murthy 15/Nov/11 00:53

          People

          • Assignee:
            Eli Collins
            Reporter:
            Eli Collins
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development