Hadoop Common
  1. Hadoop Common
  2. HADOOP-4179

Hadoop-Vaidya : Rule based performance diagnostic tool for Map/Reduce jobs

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.18.1
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      ALL

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Introduced Vaidya rule based performance diagnostic tool for Map/Reduce jobs.

      Description

      Hadoop-Vaidya is a rule based performance diagnostic tool for Map/Reduce jobs. It performs a post execution analysis of map/reduce job by parsing and collecting execution statistics through job history log and job configuration files. It runs a set of predefined tests/rules against job execution statistics to diagnose various performance problems. Each test rule detects a specific performance problem with job and provides a targeted advice to the user. This tool generates an XML report based on the evaluation results of individual test rules.

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk #683 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/683/)
          . Add Vaidya tool to analyze map/reduce job logs for performanc
          problems. (Suhas Gogate via omalley)

          Show
          Hudson added a comment - Integrated in Hadoop-trunk #683 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/683/ ) . Add Vaidya tool to analyze map/reduce job logs for performanc problems. (Suhas Gogate via omalley)
          Hide
          Owen O'Malley added a comment -

          I just committed this. Thanks, Suhas!

          Show
          Owen O'Malley added a comment - I just committed this. Thanks, Suhas!
          Hide
          Suhas Gogate added a comment -

          Can we commit this patch? some folks inquiring for it.

          Show
          Suhas Gogate added a comment - Can we commit this patch? some folks inquiring for it.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12394648/hadoop_vaidya_0200-1.patch
          against trunk revision 720698.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 21 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12394648/hadoop_vaidya_0200-1.patch against trunk revision 720698. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3653/console This message is automatically generated.
          Hide
          Suhas Gogate added a comment -

          patch submitted

          Show
          Suhas Gogate added a comment - patch submitted
          Hide
          Suhas Gogate added a comment -

          New file attached hadoop_vaidya_0200-1.patch with changes making the module named vaidya under contrib (instead of hadoop-vaidya). Corresponding changes also made in the source and documentation.

          Show
          Suhas Gogate added a comment - New file attached hadoop_vaidya_0200-1.patch with changes making the module named vaidya under contrib (instead of hadoop-vaidya). Corresponding changes also made in the source and documentation.
          Hide
          Owen O'Malley added a comment -

          Sorry, I've only glanced at the patch, but it would be better to make the contrib module 'vaidya' instead of 'hadoop-vaidya' since the hadoop is implied.

          Show
          Owen O'Malley added a comment - Sorry, I've only glanced at the patch, but it would be better to make the contrib module 'vaidya' instead of 'hadoop-vaidya' since the hadoop is implied.
          Hide
          Suhas Gogate added a comment -

          failed unit test is NOT related to this patch.

          Show
          Suhas Gogate added a comment - failed unit test is NOT related to this patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12394453/hadoop_vaidya_0200.patch
          against trunk revision 719787.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 21 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12394453/hadoop_vaidya_0200.patch against trunk revision 719787. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3636/console This message is automatically generated.
          Hide
          Suhas Gogate added a comment -

          submitting the patch

          Show
          Suhas Gogate added a comment - submitting the patch
          Hide
          Suhas Gogate added a comment -

          Found a bug.. so canceling the patch and will be resubmitting the fixed version soon.

          Show
          Suhas Gogate added a comment - Found a bug.. so canceling the patch and will be resubmitting the fixed version soon.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12394120/hadoop-vaidya-trunk.patch
          against trunk revision 719431.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 20 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12394120/hadoop-vaidya-trunk.patch against trunk revision 719431. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3615/console This message is automatically generated.
          Hide
          Suhas Gogate added a comment -

          changed the fixed version to 0.20.0 (by mistake was selected 0.19 )

          Show
          Suhas Gogate added a comment - changed the fixed version to 0.20.0 (by mistake was selected 0.19 )
          Hide
          Suhas Gogate added a comment -

          making a patch available for hadoop-vaidya in the contrib (moved it outside the chukwa as thought initially).

          Show
          Suhas Gogate added a comment - making a patch available for hadoop-vaidya in the contrib (moved it outside the chukwa as thought initially).
          Hide
          Owen O'Malley added a comment -

          I believe that the right solution is to make this project a stand-alone contrib project and handle the chukwa dependencies dynamically. Otherwise, users that aren't using chukwa won't see this tool.

          Show
          Owen O'Malley added a comment - I believe that the right solution is to make this project a stand-alone contrib project and handle the chukwa dependencies dynamically. Otherwise, users that aren't using chukwa won't see this tool.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12393268/hadoop_vaidya_trunk.patch
          against trunk revision 709609.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 20 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12393268/hadoop_vaidya_trunk.patch against trunk revision 709609. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3525/console This message is automatically generated.
          Hide
          Suhas Gogate added a comment -

          Owen, Sorry, I did not mean that Hadoop Vaidya would loose its independent identity after integration with Chukwa. Optionally It should also run outside Chukwa framework, and so I agree with you. Although based on prior discussions between Me, Runping, Mac and Milind, it appears to have better fit and more potential with Chukwa framework and so was decided to be part of Chukwa project.

          Milind/Mac/Runping, comments?

          Show
          Suhas Gogate added a comment - Owen, Sorry, I did not mean that Hadoop Vaidya would loose its independent identity after integration with Chukwa. Optionally It should also run outside Chukwa framework, and so I agree with you. Although based on prior discussions between Me, Runping, Mac and Milind, it appears to have better fit and more potential with Chukwa framework and so was decided to be part of Chukwa project. Milind/Mac/Runping, comments?
          Hide
          Owen O'Malley added a comment -

          1. I think it would add value outside of Chukwa and be usable without Chukwa. Maybe the right place is another contrib project?

          2. No. 0.19 is closed for both core and contrib.

          Show
          Owen O'Malley added a comment - 1. I think it would add value outside of Chukwa and be usable without Chukwa. Maybe the right place is another contrib project? 2. No. 0.19 is closed for both core and contrib.
          Hide
          Suhas Gogate added a comment -

          1. Although it is an independent tool right now, eventually would be integrated with Chukwa (part of health care center), providing rule based analysis of the data it collects for M/R jobs. At this time it does not use any Chukwa interface/library etc.

          2. As Milind mentioned in one of his earlier comments, I was under impression that feature freeze applies to hadoop core and not "contrib". Is it true?

          Show
          Suhas Gogate added a comment - 1. Although it is an independent tool right now, eventually would be integrated with Chukwa (part of health care center), providing rule based analysis of the data it collects for M/R jobs. At this time it does not use any Chukwa interface/library etc. 2. As Milind mentioned in one of his earlier comments, I was under impression that feature freeze applies to hadoop core and not "contrib". Is it true?
          Hide
          Owen O'Malley added a comment -

          This can't go into 0.19, because it is new functionality.

          Why is it in chukwa, rather than some where more global? Does it depend on chukwa?

          Show
          Owen O'Malley added a comment - This can't go into 0.19, because it is new functionality. Why is it in chukwa, rather than some where more global? Does it depend on chukwa?
          Hide
          Suhas Gogate added a comment -

          Making patch available

          Show
          Suhas Gogate added a comment - Making patch available
          Hide
          Suhas Gogate added a comment -

          Added patch for trunk

          Show
          Suhas Gogate added a comment - Added patch for trunk
          Hide
          Suhas Gogate added a comment -

          Patch added for branch 0.19.0

          Show
          Suhas Gogate added a comment - Patch added for branch 0.19.0
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12393197/hadoop_vaidya.0.3.patch
          against trunk revision 709609.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 20 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12393197/hadoop_vaidya.0.3.patch against trunk revision 709609. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3517/console This message is automatically generated.
          Hide
          Suhas Gogate added a comment -

          making it patch available.

          Show
          Suhas Gogate added a comment - making it patch available.
          Hide
          Suhas Gogate added a comment -

          latest patch file (hadoop_vaidya.0.3.patch) attached.

          Show
          Suhas Gogate added a comment - latest patch file (hadoop_vaidya.0.3.patch) attached.
          Hide
          Suhas Gogate added a comment -

          cancelled

          Show
          Suhas Gogate added a comment - cancelled
          Hide
          Suhas Gogate added a comment -

          patch submitted along with documentation.

          Show
          Suhas Gogate added a comment - patch submitted along with documentation.
          Hide
          Suhas Gogate added a comment -

          error applying patch. adding new patch.

          Show
          Suhas Gogate added a comment - error applying patch. adding new patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12392443/hadoop-vaidya-0.2.patch
          against trunk revision 709040.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 23 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3509/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12392443/hadoop-vaidya-0.2.patch against trunk revision 709040. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 23 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3509/console This message is automatically generated.
          Hide
          Milind Bhandarkar added a comment -

          Making it patch available.

          Show
          Milind Bhandarkar added a comment - Making it patch available.
          Hide
          Suhas Gogate added a comment -

          Hi Sachin, I sent you a detailed reply to your email couple of days back. Yes there is a possibility, although need to know few details about your familiarity with hadoop M/R and what is the start date for your project etc. We can discuss it off-line to keep JIRA updates related to technical discussion. will send you the email again today. thx. --Suhas

          Show
          Suhas Gogate added a comment - Hi Sachin, I sent you a detailed reply to your email couple of days back. Yes there is a possibility, although need to know few details about your familiarity with hadoop M/R and what is the start date for your project etc. We can discuss it off-line to keep JIRA updates related to technical discussion. will send you the email again today. thx. --Suhas
          Hide
          Milind Bhandarkar added a comment -

          Enis, I believe the feature freeze applies to hadoop core, and not contrib, right ?

          Show
          Milind Bhandarkar added a comment - Enis, I believe the feature freeze applies to hadoop core, and not contrib, right ?
          Hide
          Sachin added a comment -

          Hi Suhas, Lohith

          We are students from University of Minnesota looking for a project for our distributed systems class. We found this project to be quite interesting. We would like to work on the extension for run time analysis. How feasible is that considering we have a time frame of 4 weeks to complete our project.

          Thanks

          Sachin

          Show
          Sachin added a comment - Hi Suhas, Lohith We are students from University of Minnesota looking for a project for our distributed systems class. We found this project to be quite interesting. We would like to work on the extension for run time analysis. How feasible is that considering we have a time frame of 4 weeks to complete our project. Thanks Sachin
          Hide
          Suhas Gogate added a comment -

          Thanks Lohit, Mac & Devraj for providing the comments. Attaching the new patch incorporating comments to make it comply with hadoop coding conventions.

          Thanks.

          Show
          Suhas Gogate added a comment - Thanks Lohit, Mac & Devraj for providing the comments. Attaching the new patch incorporating comments to make it comply with hadoop coding conventions. Thanks.
          Hide
          Enis Soztutar added a comment -

          moving this from 0.19 due to feature freeze.

          Show
          Enis Soztutar added a comment - moving this from 0.19 due to feature freeze.
          Hide
          Devaraj Das added a comment -

          Comments on the coding style - Indentation needs to be fixed to conform to hadoop's coding conventions. Also the variable names are better read without the underscore prefix.

          Show
          Devaraj Das added a comment - Comments on the coding style - Indentation needs to be fixed to conform to hadoop's coding conventions. Also the variable names are better read without the underscore prefix.
          Hide
          Mac Yang added a comment -

          Looks good over all. But the @author tags should probably be removed.

          Show
          Mac Yang added a comment - Looks good over all. But the @author tags should probably be removed.
          Hide
          Milind Bhandarkar added a comment -

          Lohit,

          This is the first version of Hadoop Vaidya, which works only in the postmortem analysis mode. Later, as needed, it can be extended to do analysis of a running job too.

          P.S. Can you review the patch ? Thanks.

          Show
          Milind Bhandarkar added a comment - Lohit, This is the first version of Hadoop Vaidya, which works only in the postmortem analysis mode. Later, as needed, it can be extended to do analysis of a running job too. P.S. Can you review the patch ? Thanks.
          Hide
          Lohit Vijayarenu added a comment -

          This would be very useful to have. Can jobdetails.jsp or jobtracker.jsp (may be via StatusHttpServlet) serve this analysis after the job is done? That will help users to check how job performed directly from webUI.
          PS : Definition of Vaidya

          Show
          Lohit Vijayarenu added a comment - This would be very useful to have. Can jobdetails.jsp or jobtracker.jsp (may be via StatusHttpServlet) serve this analysis after the job is done? That will help users to check how job performed directly from webUI. PS : Definition of Vaidya
          Hide
          Suhas Gogate added a comment -

          This JIRA takes care of post execution diagnosis of map/reduce job. The linked jira (HADOOP-3956) would extend it further for runtime progress analysis of Map/Reduce jobs.

          Show
          Suhas Gogate added a comment - This JIRA takes care of post execution diagnosis of map/reduce job. The linked jira ( HADOOP-3956 ) would extend it further for runtime progress analysis of Map/Reduce jobs.
          Hide
          Suhas Gogate added a comment -

          patch added

          Show
          Suhas Gogate added a comment - patch added

            People

            • Assignee:
              Suhas Gogate
              Reporter:
              Suhas Gogate
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development