Hadoop Common
  1. Hadoop Common
  2. HADOOP-7424

Log an error if the topology script doesn't handle multiple args

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      ScriptBasedMapping#resolve currently warns and returns null if it passes n arguments to the topology script and gets back a different number of resolutions. This indicates a bug in the topology script (or it's input) and therefore should be an error.

      // invalid number of entries returned by the script
      LOG.warn("Script " + scriptName + " returned "
         + Integer.toString(m.size()) + " values when "
         + Integer.toString(names.size()) + " were expected.");
      return null;
      

      There's only one place in Hadoop (FSNamesystem init) where we pass multiple arguments to the topology script, and it only done for performance (to trigger resolution/caching of all the hosts in the includes file on startup). So currently a topology script that doesn't handle multiple arguments just means the initial cache population doesn't work.

      1. HADOOP-7424.patch
        1 kB
        Uma Maheswara Rao G

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          111d 1h 20m 1 Uma Maheswara Rao G 13/Oct/11 01:43
          Patch Available Patch Available Resolved Resolved
          15d 1h 42m 1 Eli Collins 28/Oct/11 03:26
          Resolved Resolved Closed Closed
          129d 23m 1 Arun C Murthy 05/Mar/12 02:49
          Gavin made changes -
          Link This issue is depended upon by HADOOP-7863 [ HADOOP-7863 ]
          Gavin made changes -
          Link This issue blocks HADOOP-7863 [ HADOOP-7863 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 0.23.1 [ 12318884 ]
          Fix Version/s 0.24.0 [ 12317652 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #912 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/912/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #912 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/912/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #108 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/108/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #108 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/108/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #879 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/879/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #879 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/879/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #92 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/92/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #92 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/92/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1355 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1355/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1355 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1355/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #233 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/233/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #233 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/233/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1331 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1331/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1331 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1331/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1405 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1405/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1405 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1405/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207614 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #219 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/219/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #219 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/219/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #222 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/222/)
          HADOOP-7863: apply HADOOP-7424 to 0.23.1

          stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #222 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/222/ ) HADOOP-7863 : apply HADOOP-7424 to 0.23.1 stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1207611 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Steve Loughran made changes -
          Link This issue blocks HADOOP-7863 [ HADOOP-7863 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #846 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/846/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #846 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/846/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #876 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/876/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #876 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/876/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1182 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1182/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1182 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1182/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1259 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1259/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1259 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1259/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1202 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1202/)
          HADOOP-7424. Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G

          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1202 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1202/ ) HADOOP-7424 . Log an error if the topology script doesn't handle multiple args. Contributed by Uma Maheswara Rao G eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190139 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/ScriptBasedMapping.java
          Eli Collins made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Assignee Uma Maheswara Rao G [ umamaheswararao ]
          Fix Version/s 0.24.0 [ 12317652 ]
          Resolution Fixed [ 1 ]
          Hide
          Eli Collins added a comment -

          I've committed this. Thanks Uma!

          Show
          Eli Collins added a comment - I've committed this. Thanks Uma!
          Hide
          Eli Collins added a comment -

          +1 (no new test needed since we're just changing a warn to an error)

          Show
          Eli Collins added a comment - +1 (no new test needed since we're just changing a warn to an error)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12498807/HADOOP-7424.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12498807/HADOOP-7424.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/292//console This message is automatically generated.
          Uma Maheswara Rao G made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Uma Maheswara Rao G made changes -
          Attachment HADOOP-7424.patch [ 12498807 ]
          Hide
          Uma Maheswara Rao G added a comment -

          Small patch as per the Eli suggestions and cleaned some of the unused imports.

          Show
          Uma Maheswara Rao G added a comment - Small patch as per the Eli suggestions and cleaned some of the unused imports.
          Eli Collins made changes -
          Field Original Value New Value
          Tags newbie
          Labels newbie
          Eli Collins created issue -

            People

            • Assignee:
              Uma Maheswara Rao G
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development