Hadoop Common
  1. Hadoop Common
  2. HADOOP-2911

[HOD] Make the information printed by allocate and info commands less verbose and clearer

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.16.0
    • Fix Version/s: 0.16.1
    • Component/s: contrib/hod
    • Labels:
      None

      Description

      Currently hod prints the following information as part of the allocate command at an 'INFO' verbose level:

      Service Registry started
      Torque job id
      Ringmaster ..
      HDFS UI at ..
      Mapred UI at ..

      The "service registry started" line is misleading and isn't very useful. This should be removed.
      Torque job id is misleading in terminology because job id also clashes with Mapred "job id". The proposal is to make it something like 'cluster id' or something similar.
      Ringmaster information is useful for debugging, but not for regular users.

      Similar changes should be made to the hod info command as well. The naming should be clear and consistent. There, we print all of the above information and in addition, also print the number of nodes as min,max. Because we don't really support the min,max number of nodes, we can remove that and probably print only the number of nodes allocated.

      All information must continue to be printed at a higher verbosity level.

      1. hadoop-2911
        7 kB
        Vinod Kumar Vavilapalli
      2. hadoop-2911.1
        7 kB
        Hemanth Yamijala

        Activity

        Hide
        Nigel Daley added a comment -

        Here are some other typo's that could also be fixed as part of this:

        > % hod -o "allocate ~/tmp 3"
        > INFO - Service Registry Started.
        > INFO - Hod Job successfully submitted. JobId : 9277.gs203000.inktomisearch.com.
        > INFO - Ringmaster at : http://gs203031:36333/.
        > INFO - HDFS UI on http://gs203031.inktomisearch.com:57972
        > INFO - Mapred UI on http://gs203033.inktomisearch.com:54712

        Remove periods from the end of lines
        on -> at

        ----------------------

        > % hod -o "allocate ~/tmp 300"
        > INFO - Service Registry Started.

        lower case "Started"

        > CRITICAL - Caught signal 2.
        > INFO - Hod Interrupted. Cleaning up and exitting

        Interrupted. -> interrupted.
        exitting -> exiting.

        > INFO - Job 9278.gs203000.inktomisearch.com qdelled.

        qdelled. -> deleted from queue.

        > CRITICAL - op: allocate ~/tmp 300 failed because of an process interrupt.

        an -> a

        Show
        Nigel Daley added a comment - Here are some other typo's that could also be fixed as part of this: > % hod -o "allocate ~/tmp 3" > INFO - Service Registry Started. > INFO - Hod Job successfully submitted. JobId : 9277.gs203000.inktomisearch.com. > INFO - Ringmaster at : http://gs203031:36333/ . > INFO - HDFS UI on http://gs203031.inktomisearch.com:57972 > INFO - Mapred UI on http://gs203033.inktomisearch.com:54712 Remove periods from the end of lines on -> at ---------------------- > % hod -o "allocate ~/tmp 300" > INFO - Service Registry Started. lower case "Started" > CRITICAL - Caught signal 2. > INFO - Hod Interrupted. Cleaning up and exitting Interrupted. -> interrupted. exitting -> exiting. > INFO - Job 9278.gs203000.inktomisearch.com qdelled. qdelled. -> deleted from queue. > CRITICAL - op: allocate ~/tmp 300 failed because of an process interrupt. an -> a
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Attaching patch. In addition to suggested changes, made changes to display "hadoop-site.xml at <clusterdir>" in both info and allocate operations' output.

        Show
        Vinod Kumar Vavilapalli added a comment - Attaching patch. In addition to suggested changes, made changes to display "hadoop-site.xml at <clusterdir>" in both info and allocate operations' output.
        Hide
        Hemanth Yamijala added a comment -

        Trivial changes to Vinod's patch. Modified 2 strings so they read the same in the allocate and info commands. Otherwise +1 for the patch. Going to run past Hudson.

        Show
        Hemanth Yamijala added a comment - Trivial changes to Vinod's patch. Modified 2 strings so they read the same in the allocate and info commands. Otherwise +1 for the patch. Going to run past Hudson.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12377148/hadoop-2911.1
        against trunk revision 619744.

        @author +1. The patch does not contain any @author tags.

        tests included -1. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new javac compiler warnings.

        release audit +1. The applied patch does not generate any new release audit warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests -1. The patch failed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12377148/hadoop-2911.1 against trunk revision 619744. @author +1. The patch does not contain any @author tags. tests included -1. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/1894/console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        HOD cannot have unittests till HADOOP-2775 is resolved.

        Core tests failed are not due to this patch.

        Patch is committable.

        Show
        Vinod Kumar Vavilapalli added a comment - HOD cannot have unittests till HADOOP-2775 is resolved. Core tests failed are not due to this patch. Patch is committable.
        Hide
        Nigel Daley added a comment -

        I just commited this. Thanks Vinod!

        Show
        Nigel Daley added a comment - I just commited this. Thanks Vinod!
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #421 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/421/ )
        Hide
        Karam Singh added a comment -

        Verified that ouput of hod command has been updated

        Show
        Karam Singh added a comment - Verified that ouput of hod command has been updated

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development