Some job specific configuration properties(but not complete) mentioned in MAPREDUCE-1658 that are to be brought into trace file are:

• mapreduce.map.speculative, mapreduce.reduce.speculative
• mapreduce.job.reduce.slowstart.completedmaps
• mapreduce.task.io.sort.factor, mapreduce.task.io.sort.mb, mapreduce.map.sort.spill.percent
• mapreduce.reduce.shuffle.connect.timeout, mapreduce.reduce.shuffle.read.timeout
• mapreduce.reduce.shuffle.merge.percent, mapreduce.reduce.shuffle.input.buffer.percent
• mapreduce.reduce.merge.inmem.threshold

Resolved MAPREDUCE-1658 as duplicate of this JIRA.

Attaching the patch for apache trunk. This patch ensures that all job properties are saved in the json file under "jobProperties" tag.

ant test-patch results

findbugs are not related to this patch.

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12475671/mr-2153-test-patch-results.txt
against trunk revision 1090390.

-1 @author. The patch appears to contain 3 @author tags which the Hadoop community has agreed to not allow in code contributions.

+1 tests included. The patch appears to include 503 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/164//console

This message is automatically generated.

Uploading the same patch for running via Hudson

Cancelling as Hudson picked up the wrong file.

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12476085/MapReduce-2153-trunk.patch
against trunk revision 1090390.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 15 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

-1 javac. The applied patch generated 2247 javac compiler warnings (more than the trunk's current 2244 warnings).

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

+1 system test framework. The patch passed system test framework compile.

Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/165//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/165//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/165//console

This message is automatically generated.

Fixed the javac warnings in earlier patch

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12476206/MR-2153-patch.txt
against trunk revision 1090390.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 15 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed core unit tests.

-1 contrib tests. The patch failed contrib unit tests.

+1 system test framework. The patch passed system test framework compile.

Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/166//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/166//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/166//console

This message is automatically generated.

The two testcases that failed are
1. org.apache.hadoop.mapred.TestFairSchedulerSystem.testFairSchedulerSystem
2. org.apache.hadoop.raid.TestBlockFixerDistConcurrency.testConcurrentJobs

They are not related to this patch. I will go ahead and commit this.

Marking those methods deprecated and removing/modifying the calls to those methods seems to be a better way — for the issues identified by the "javac warnings".

Discussed offline with Amar. There seems to be no way to mark those methods as deprecated and remove references and not break backwards compatibility. Backward compatibility in the sense that supporting older trace files which contain fields and don't contain JobProperties.

So let us go ahead keeping those existing setters and getters as they are for some time(few releases) and make sure that more setters and getters are not added from now on.

+1 for the latest patch.

I just committed this. Thanks Rajesh and Ravi!