Hadoop Common
  1. Hadoop Common
  2. HADOOP-2982

[HOD] checknodes should look for free nodes without the jobs attribute

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.16.0
    • Fix Version/s: 0.17.0
    • Component/s: contrib/hod
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      The number of free nodes in the cluster is computed using a better algorithm that filters out inconsistencies in node status as reported by Torque.

      Description

      Modify the checknodes logic to compute number of available nodes by looking for the number of free nodes without a jobs attribute. This is more accurate than current computation.

      1. 2982.patch
        0.8 kB
        Hemanth Yamijala

        Activity

        Hide
        Hemanth Yamijala added a comment -

        Marking as blocker, as this will be needed for our Ops team.

        Show
        Hemanth Yamijala added a comment - Marking as blocker, as this will be needed for our Ops team.
        Hide
        Hemanth Yamijala added a comment -

        The attached file modifies the way we compute free nodes. The previous computation had a problem in some corner cases. When there are runaway jobs, the pbsnodes utility reports that some nodes are free, but the pbsmoms report that jobs are running. Such nodes should not be counted as free, as jobs will not successfully run on them. The patch discards such nodes.

        Show
        Hemanth Yamijala added a comment - The attached file modifies the way we compute free nodes. The previous computation had a problem in some corner cases. When there are runaway jobs, the pbsnodes utility reports that some nodes are free, but the pbsmoms report that jobs are running. Such nodes should not be counted as free, as jobs will not successfully run on them. The patch discards such nodes.
        Hide
        Hemanth Yamijala added a comment -

        Running past Hudson. No unit test for this patch, as it is a script file change which relies on an external utility.

        Show
        Hemanth Yamijala added a comment - Running past Hudson. No unit test for this patch, as it is a script file change which relies on an external utility.
        Hide
        Rajiv Chittajallu added a comment -

        +1 for the patch.

        Show
        Rajiv Chittajallu added a comment - +1 for the patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12378622/2982.patch
        against trunk revision 619744.

        @author +1. The patch does not contain any @author tags.

        tests included -1. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new javac compiler warnings.

        release audit +1. The applied patch does not generate any new release audit warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests +1. The patch passed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12378622/2982.patch against trunk revision 619744. @author +1. The patch does not contain any @author tags. tests included -1. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2063/console This message is automatically generated.
        Hide
        Devaraj Das added a comment -

        I just committed this. Thanks, Hemanth!

        Show
        Devaraj Das added a comment - I just committed this. Thanks, Hemanth!
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #444 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/444/ )

          People

          • Assignee:
            Hemanth Yamijala
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development