Robert Kanter, thanks for taking care of this issue. We could have a MR framework-level config, such as "mapreduce.jobhistory.timeline-service.enabled", and it can also be overwritten by each individual MR job by specifying it in CLI args. Only when this flag is set, the logic that history events are posted to the timeline server will be executed.

I added the config to MRJobConfig and named it mapreduce.job.timeline-service.enabled and defaults to true.

This requires MAPREDUCE-5933.

Thanks for the patch, Robert Kanter! Some minor comments about it.

1. It's not necessary var. The client is only constructed when the flag is set. Then we try to use timeline client, just check null first.

+  private boolean timelineEnabled;

2. Can you add the config to mapred-default.xml and add a description?

I forgot mentioning how about using false as the default value? MR is a long-live and widely used framework, let's be more conservative when introducing a new feature it.

Haven't looked at all of the patch, but one comment: How about mapreduce.job.emit-timeline-data (or something similar) instead of mapreduce.job.timeline-service.enabled? If you agree, similarly rename the constants.

Let's say mapreduce.job.emit-timeline-data.enabled?

mapreduce.job.emit-timeline-data is already a boolean, no need for enabled. Otherwise we can follow RM's config and say mapreduce.job.timeline-data-publisher.enabled?

mapreduce.job.emit-timeline-data

Thanks for clarification. I used to think "emit-timeline-data" means a sub module of job. In MAPREDUCE-5933, we actually didn't introduce an obvious submodule, so let's use mapreduce.job.emit-timeline-data = false.

The new patch gets rid of the boolean, renames the config property to mapreduce.job.emit-timeline-data (which I think is more clear), defaults it to false, and adds it to mapped-site.xml.

+1 for the last patch. Do some cosmetic work: fix the config description and add a log. Will commit this patch after MAPREDUCE-5933

Again, Jenkins is temporally broken. The last patch is minor change upon the prior one. Will go ahead to commit it directly.

Committed the patch to trunk, branch-2 and branch-2.6. Thanks Robert for the patch, and Vinod for input about config name!

SUCCESS: Integrated in Hadoop-trunk-Commit #6363 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6363/)
MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)

• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
• hadoop-mapreduce-project/CHANGES.txt
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

SUCCESS: Integrated in Hadoop-Yarn-trunk #726 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/726/)
MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)

• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
• hadoop-mapreduce-project/CHANGES.txt
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1940 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1940/)
MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)

• hadoop-mapreduce-project/CHANGES.txt
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java

SUCCESS: Integrated in Hadoop-Hdfs-trunk #1915 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1915/)
MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. (zjshen: rev 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)

• hadoop-mapreduce-project/CHANGES.txt
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
• hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java