Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3907

Document entries mapred-default.xml for the jobhistory server.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The following configuration properties are documented in http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html#Running_Hadoop_in_Secure_Mode

      • mapreduce.jobhistory.address
      • mapreduce.jobhistory.keytab
      • mapreduce.jobhistory.principal

      Create a hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/resources/mapred-default.xml that documents these properties and provides default values.

      1. MAPREDUCE-3907.patch
        1 kB
        Harsh J
      2. MAPREDUCE-3907.patch
        1 kB
        Harsh J
      3. MAPREDUCE-3907.patch
        2 kB
        Eugene Koontz

        Issue Links

          Activity

          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12515827/MAPREDUCE-3907.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1989//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1989//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515827/MAPREDUCE-3907.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1989//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1989//console This message is automatically generated.
          Hide
          Harsh J added a comment -

          Hi Eugene,

          The patch's additions look good, but the location is wrong. We can rather add it to ./hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml itself, as thats the real place thats to be used (even job history server uses the MR core).

          I will update your patch to add it in here and commit it in after that. Thank you!

          Show
          Harsh J added a comment - Hi Eugene, The patch's additions look good, but the location is wrong. We can rather add it to ./hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml itself, as thats the real place thats to be used (even job history server uses the MR core). I will update your patch to add it in here and commit it in after that. Thank you!
          Hide
          Harsh J added a comment -

          I'll do the above after MAPREDUCE-3223 is checked in, to avoid conflicts.

          Show
          Harsh J added a comment - I'll do the above after MAPREDUCE-3223 is checked in, to avoid conflicts.
          Hide
          Eugene Koontz added a comment -

          Thanks Harsh J!

          Show
          Eugene Koontz added a comment - Thanks Harsh J!
          Hide
          Harsh J added a comment -

          New patch that is updated for current trunk state and the proper mapred-default.xml

          Will commit this in once Jenkins gives its +1.

          Show
          Harsh J added a comment - New patch that is updated for current trunk state and the proper mapred-default.xml Will commit this in once Jenkins gives its +1.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12535582/MAPREDUCE-3907.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2557//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535582/MAPREDUCE-3907.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2557//console This message is automatically generated.
          Hide
          Harsh J added a comment -

          Weird failure of the bot. Perhaps the cause was the -p1 patch instead of -p0 but I thought those were handled properly. Here's a proper prefix-less patch to retry.

          Show
          Harsh J added a comment - Weird failure of the bot. Perhaps the cause was the -p1 patch instead of -p0 but I thought those were handled properly. Here's a proper prefix-less patch to retry.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12535586/MAPREDUCE-3907.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2558//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535586/MAPREDUCE-3907.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2558//console This message is automatically generated.
          Hide
          Harsh J added a comment -

          Well the HS tests all pass with this change installed locally, so I am going ahead and committing it in:

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryEntities
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.989 sec
          Running org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryEvents
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.395 sec
          Running org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.643 sec
          Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp
          Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.404 sec
          Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServices
          Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.925 sec
          Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesAttempts
          Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.896 sec
          Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobConf
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.705 sec
          Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobs
          Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.366 sec
          Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobsQuery
          Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.372 sec
          Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesTasks
          Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.96 sec
          
          Results :
          
          Tests run: 122, Failures: 0, Errors: 0, Skipped: 0
          
          Show
          Harsh J added a comment - Well the HS tests all pass with this change installed locally, so I am going ahead and committing it in: ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryEntities Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.989 sec Running org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryEvents Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.395 sec Running org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryParsing Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.643 sec Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.404 sec Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServices Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.925 sec Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesAttempts Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.896 sec Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobConf Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.705 sec Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobs Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.366 sec Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobsQuery Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.372 sec Running org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesTasks Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.96 sec Results : Tests run: 122, Failures: 0, Errors: 0, Skipped: 0
          Hide
          Harsh J added a comment -

          Committed to branch-2 and trunk. Thank you for your many contributions Eugene!

          Show
          Harsh J added a comment - Committed to branch-2 and trunk. Thank you for your many contributions Eugene!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1098 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1098/)
          MAPREDUCE-3907. Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892)

          Result = FAILURE
          harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1098 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1098/ ) MAPREDUCE-3907 . Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2434 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2434/)
          MAPREDUCE-3907. Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892)

          Result = SUCCESS
          harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2434 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2434/ ) MAPREDUCE-3907 . Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2502 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2502/)
          MAPREDUCE-3907. Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892)

          Result = SUCCESS
          harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2502 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2502/ ) MAPREDUCE-3907 . Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2452 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2452/)
          MAPREDUCE-3907. Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892)

          Result = FAILURE
          harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2452 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2452/ ) MAPREDUCE-3907 . Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1131 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1131/)
          MAPREDUCE-3907. Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892)

          Result = SUCCESS
          harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1131 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1131/ ) MAPREDUCE-3907 . Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Hide
          Harsh J added a comment -

          This has been committed to branch-0.23 as well:

          Integrated in Hadoop-Hdfs-0.23-Build #309 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/309/)
          Backport HADOOP-3907 from trunk (svn merge -c 1358892). (harsh) (Revision 1359179)
          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359179
          Files :
          /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

          But the commit message accidentally notified HADOOP-3907.

          (Just noting here for posterity's sake)

          Show
          Harsh J added a comment - This has been committed to branch-0.23 as well: Integrated in Hadoop-Hdfs-0.23-Build #309 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/309/ ) Backport HADOOP-3907 from trunk (svn merge -c 1358892). (harsh) (Revision 1359179) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359179 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml But the commit message accidentally notified HADOOP-3907 . (Just noting here for posterity's sake)
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2508 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2508/)
          MAPREDUCE-3907. Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892)

          Result = SUCCESS
          harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2508 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2508/ ) MAPREDUCE-3907 . Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2441 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2441/)
          MAPREDUCE-3907. Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892)

          Result = SUCCESS
          harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2441 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2441/ ) MAPREDUCE-3907 . Document entries mapred-default.xml for the jobhistory server. Contributed by Eugene Koontz. (harsh) (Revision 1358892) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1358892 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

            People

            • Assignee:
              Eugene Koontz
              Reporter:
              Eugene Koontz
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development