Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0, 1.2.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Allow ServicePlugins (see HADOOP-5257) for the JobTracker.

      1. MAPREDUCE-461.branch-1.patch
        6 kB
        Brandon Li
      2. MAPREDUCE-461.patch
        6 kB
        Tom White
      3. sp-jt-1.diff
        2 kB
        Fredrik Hedberg

        Issue Links

          Activity

          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12405064/sp-jt-1.diff
          against trunk revision 763728.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12405064/sp-jt-1.diff against trunk revision 763728. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/176/console This message is automatically generated.
          Hide
          Fredrik Hedberg added a comment -

          AFAIK, the failed tests are not related to this issue. I don't think this patch warrants any new unit tests.

          Show
          Fredrik Hedberg added a comment - AFAIK, the failed tests are not related to this issue. I don't think this patch warrants any new unit tests.
          Hide
          Amar Kamat added a comment -

          Fredrik, the patch looks good. Few minor comments.

          1. Can you add 'mapred.jobtracker.plugins' to mapred-default.xml and document it there.
          2. LOG.warn("ServicePlugin " + p + " could not be started", t);

            and

            LOG.warn("ServicePlugin " + p + " could not be stopped", t);

            Here p will log junk info about the object (Object.toString()). Either we can simply dump the exception or else define ServicePlugin.getName().

          Rest looks good.

          Show
          Amar Kamat added a comment - Fredrik, the patch looks good. Few minor comments. Can you add 'mapred.jobtracker.plugins' to mapred-default.xml and document it there. LOG.warn("ServicePlugin " + p + " could not be started", t); and LOG.warn("ServicePlugin " + p + " could not be stopped", t); Here p will log junk info about the object (Object.toString()). Either we can simply dump the exception or else define ServicePlugin.getName() . Rest looks good.
          Hide
          Devaraj Das added a comment -

          Could you please write a unit test for this? Thanks!

          Show
          Devaraj Das added a comment - Could you please write a unit test for this? Thanks!
          Hide
          Sharad Agarwal added a comment -

          Waiting for unit test.

          Show
          Sharad Agarwal added a comment - Waiting for unit test.
          Hide
          Amr Awadallah added a comment -

          Fredrik, will you write a unit test for it?

          Show
          Amr Awadallah added a comment - Fredrik, will you write a unit test for it?
          Hide
          Allen Wittenauer added a comment -

          It looks like Cloudera has this patch in their distribution, yet it doesn't appear to have been committed upstream. What is the status? Are we still waiting for a unit test?

          Show
          Allen Wittenauer added a comment - It looks like Cloudera has this patch in their distribution, yet it doesn't appear to have been committed upstream. What is the status? Are we still waiting for a unit test?
          Hide
          Tom White added a comment -

          Here's an updated patch which includes a unit test. It also addresses Amar's feedback.

          Show
          Tom White added a comment - Here's an updated patch which includes a unit test. It also addresses Amar's feedback.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12459224/MAPREDUCE-461.patch
          against trunk revision 1075422.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/98//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/98//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/98//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12459224/MAPREDUCE-461.patch against trunk revision 1075422. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/98//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/98//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/98//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          +1, looks good to me

          Show
          Todd Lipcon added a comment - +1, looks good to me
          Hide
          Tom White added a comment -

          I've just committed this. Thanks, Fredrik!

          Show
          Tom White added a comment - I've just committed this. Thanks, Fredrik!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #678 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/678/)
          MAPREDUCE-461. Enable ServicePlugins for the JobTracker. Contributed by Fredrik Hedberg.

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #678 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/678/ ) MAPREDUCE-461 . Enable ServicePlugins for the JobTracker. Contributed by Fredrik Hedberg.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #682 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/682/)
          MAPREDUCE-461. Enable ServicePlugins for the JobTracker. Contributed by Fredrik Hedberg.

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #682 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/682/ ) MAPREDUCE-461 . Enable ServicePlugins for the JobTracker. Contributed by Fredrik Hedberg.
          Hide
          Brandon Li added a comment -

          Uploaded a patch to back port this feature to branch-1.

          Show
          Brandon Li added a comment - Uploaded a patch to back port this feature to branch-1.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Brandon, the patch for branch-1 looks good. But it doesn't apply against 1.1, can you please update? Tx.

          Show
          Vinod Kumar Vavilapalli added a comment - Brandon, the patch for branch-1 looks good. But it doesn't apply against 1.1, can you please update? Tx.
          Hide
          Brandon Li added a comment -

          @Vinod, thanks for reviewing. The patch is actually for branch-1, not 1.1.

          Show
          Brandon Li added a comment - @Vinod, thanks for reviewing. The patch is actually for branch-1, not 1.1.
          Hide
          Brandon Li added a comment -

          Test-patch result of the branch-1 patch:

          -1 overall.  
              +1 @author.  The patch does not contain any @author tags.
              +1 tests included.  The patch appears to include 2 new or modified tests.
              +1 javadoc.  The javadoc tool did not generate any warning messages.
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
              -1 findbugs.  The patch appears to introduce 196 new Findbugs (version 2.0.0) warnings.
          

          The patch doesn't introduce new findbuf warnings.

          Show
          Brandon Li added a comment - Test-patch result of the branch-1 patch: -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 196 new Findbugs (version 2.0.0) warnings. The patch doesn't introduce new findbuf warnings.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Just committed this to branch-1. Thanks for the backport, Brandon!

          Show
          Vinod Kumar Vavilapalli added a comment - Just committed this to branch-1. Thanks for the backport, Brandon!

            People

            • Assignee:
              Fredrik Hedberg
              Reporter:
              Fredrik Hedberg
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development