Hadoop Common
  1. Hadoop Common
  2. HADOOP-7424

Log an error if the topology script doesn't handle multiple args

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      ScriptBasedMapping#resolve currently warns and returns null if it passes n arguments to the topology script and gets back a different number of resolutions. This indicates a bug in the topology script (or it's input) and therefore should be an error.

      // invalid number of entries returned by the script
      LOG.warn("Script " + scriptName + " returned "
         + Integer.toString(m.size()) + " values when "
         + Integer.toString(names.size()) + " were expected.");
      return null;
      

      There's only one place in Hadoop (FSNamesystem init) where we pass multiple arguments to the topology script, and it only done for performance (to trigger resolution/caching of all the hosts in the includes file on startup). So currently a topology script that doesn't handle multiple arguments just means the initial cache population doesn't work.

      1. HADOOP-7424.patch
        1 kB
        Uma Maheswara Rao G

        Issue Links

          Activity

          Hide
          Uma Maheswara Rao G added a comment -

          Small patch as per the Eli suggestions and cleaned some of the unused imports.

          Show
          Uma Maheswara Rao G added a comment - Small patch as per the Eli suggestions and cleaned some of the unused imports.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12498807/HADOOP-7424.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12498807/HADOOP-7424.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          +1 (no new test needed since we're just changing a warn to an error)

          Show
          Eli Collins added a comment - +1 (no new test needed since we're just changing a warn to an error)
          Hide
          Eli Collins added a comment -

          I've committed this. Thanks Uma!

          Show
          Eli Collins added a comment - I've committed this. Thanks Uma!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1202 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1202/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1202 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1202/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1259 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1259/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1259 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1259/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1182 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1182/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1182 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1182/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #876 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/876/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #876 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/876/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #846 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/846/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #846 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/846/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #222 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/222/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #222 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/222/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #219 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/219/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #219 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/219/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1405 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1405/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1405 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1405/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1331 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1331/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1331 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1331/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #233 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/233/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #233 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/233/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1355 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1355/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1355 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1355/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #92 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/92/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #92 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/92/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #879 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/879/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #879 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/879/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #108 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/108/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #108 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/108/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #912 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/912/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #912 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/912/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

            People

            • Assignee:
              Uma Maheswara Rao G
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development