Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1321

If service port and main port are the same, there is no clear log message explaining the issue.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: namenode
    • Labels:
    • Release Note:
      Added a check to match the sure RPC and HTTP Port's on the NameNode were not set to the same value, otherwise an IOException is throw with the appropriate message.

      Description

      With the introduction of a service port to the namenode, there is now a chance for user error to set the two port equal. This will cause the namenode to fail to start up. It would be nice if there was a log message explaining the port clash. Or just treat things as if the service port was not specified.

      1. HDFS-1321.patch
        5 kB
        Jim Plush
      2. HDFS-1321-take2.txt
        5 kB
        Jim Plush
      3. HDFS-1321-take3.txt
        5 kB
        Jim Plush
      4. HDFS-1321-take4.txt
        5 kB
        Jim Plush

        Issue Links

          Activity

          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12483619/HDFS-1321.patch
          against trunk revision 1138645.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/828//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483619/HDFS-1321.patch against trunk revision 1138645. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/828//console This message is automatically generated.
          Hide
          Jim Plush added a comment -

          did a hard reset and re-applied the changes to try and get a cleaner patch file

          Show
          Jim Plush added a comment - did a hard reset and re-applied the changes to try and get a cleaner patch file
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12483625/HDFS-1321-take2.txt
          against trunk revision 1138645.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/829//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/829//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/829//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483625/HDFS-1321-take2.txt against trunk revision 1138645. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/829//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/829//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/829//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Hey Jim, patch looks pretty good. Just a few small nits:

          1. The method to check the configs is called "VERIFYConfigurationSettings" whereas the test is called "TestConfigurationVALIDATIONS". I think these should be consistent, and I prefer the word "validate" in this context.
          2. No need for the StringBuilder in the case of this error - just break up the message onto a few lines, e.g.:

            "This is an " +
            "error message.";


          3. Rather than "HttpServerAddress Port" and "RPC Server Address Port" in the error message, I'd recommend using the actual config keys, which are defined in the DFSConfigKeys class.
          4. In the test, I'd recommend catching IOException explicitly, and maybe even verifying the contents of the message string a little bit. We wouldn't want this test to pass if a NullPointerException or an unrelated IOException were thrown.
          5. The indentation is off throughout the patch. The Hadoop code base uses 2-space indentation, no tabs.
          Show
          Aaron T. Myers added a comment - Hey Jim, patch looks pretty good. Just a few small nits: The method to check the configs is called "VERIFYConfigurationSettings" whereas the test is called "TestConfigurationVALIDATIONS". I think these should be consistent, and I prefer the word "validate" in this context. No need for the StringBuilder in the case of this error - just break up the message onto a few lines, e.g.: "This is an " + "error message."; Rather than "HttpServerAddress Port" and "RPC Server Address Port" in the error message, I'd recommend using the actual config keys, which are defined in the DFSConfigKeys class. In the test, I'd recommend catching IOException explicitly, and maybe even verifying the contents of the message string a little bit. We wouldn't want this test to pass if a NullPointerException or an unrelated IOException were thrown. The indentation is off throughout the patch. The Hadoop code base uses 2-space indentation, no tabs.
          Hide
          Jim Plush added a comment -

          Adding in Aaron's comments regarding the formatting and coding guideline changes

          Show
          Jim Plush added a comment - Adding in Aaron's comments regarding the formatting and coding guideline changes
          Hide
          Jim Plush added a comment -

          thanks for the info Aaron, those are all definitely reasonable requirements and I should have caught those on the first round. Let me know if this patch takes care of those issues.

          Show
          Jim Plush added a comment - thanks for the info Aaron, those are all definitely reasonable requirements and I should have caught those on the first round. Let me know if this patch takes care of those issues.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12483767/HDFS-1321-take3.txt
          against trunk revision 1139397.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/833//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/833//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/833//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483767/HDFS-1321-take3.txt against trunk revision 1139397. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/833//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/833//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/833//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Hey Jim, latest patch is almost there. Two tiny nits:

          1. I should've been more clear, but I think the error message should include the values of those config options. i.e. I think the error message should end up including "dfs.namenode.http-address" and "dfs.namenode.rpc-address" since those are what are actually put in the config files. Users won't know what DFS_NAMENODE_HTTP_ADDRESS_KEY and DFS_NAMENODE_RPC_ADDRESS_KEY mean.
          2. In testThatDifferentRPCandHttpPortsAreOK, there's no need to catch the exception and explicitly call Assert.fail(...). If an exception gets thrown by a test method, that will indicate failure to the testing framework automatically.

          +1 after these issues are addressed.

          Show
          Aaron T. Myers added a comment - Hey Jim, latest patch is almost there. Two tiny nits: I should've been more clear, but I think the error message should include the values of those config options. i.e. I think the error message should end up including "dfs.namenode.http-address" and "dfs.namenode.rpc-address" since those are what are actually put in the config files. Users won't know what DFS_NAMENODE_HTTP_ADDRESS_KEY and DFS_NAMENODE_RPC_ADDRESS_KEY mean. In testThatDifferentRPCandHttpPortsAreOK , there's no need to catch the exception and explicitly call Assert.fail(...) . If an exception gets thrown by a test method, that will indicate failure to the testing framework automatically. +1 after these issues are addressed.
          Hide
          Jim Plush added a comment -

          4 times the charm?
          ok Hopefully addressed your last two issues, the assert fail was removed from the test and the logline now reads as:

          "dfs.namenode.rpc-address (localhost/127.0.0.1:9000) and dfs.namenode.http-address (/127.0.0.1:9000) configuration keys are bound to the same port, unable to start NameNode. Port: 9000"

          Show
          Jim Plush added a comment - 4 times the charm? ok Hopefully addressed your last two issues, the assert fail was removed from the test and the logline now reads as: "dfs.namenode.rpc-address (localhost/127.0.0.1:9000) and dfs.namenode.http-address (/127.0.0.1:9000) configuration keys are bound to the same port, unable to start NameNode. Port: 9000"
          Hide
          Aaron T. Myers added a comment -

          +1, the patch looks good to me. Will commit pending clean Hudson results.

          Show
          Aaron T. Myers added a comment - +1, the patch looks good to me. Will commit pending clean Hudson results.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12483786/HDFS-1321-take4.txt
          against trunk revision 1139397.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/835//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/835//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/835//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12483786/HDFS-1321-take4.txt against trunk revision 1139397. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/835//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/835//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/835//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          I've just committed this.

          Thanks a lot for the contribution, Jim!

          Show
          Aaron T. Myers added a comment - I've just committed this. Thanks a lot for the contribution, Jim!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #759 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/759/)
          HDFS-1321. If service port and main port are the same, there is no clear log message explaining the issue. (Jim Plush via atm)

          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139473
          Files :

          • /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestValidateConfigurationSettings.java
          • /hadoop/common/trunk/hdfs/CHANGES.txt
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #759 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/759/ ) HDFS-1321 . If service port and main port are the same, there is no clear log message explaining the issue. (Jim Plush via atm) atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139473 Files : /hadoop/common/trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestValidateConfigurationSettings.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #707 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/707/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #707 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/707/ )

            People

            • Assignee:
              Jim Plush
              Reporter:
              gary murry
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development