Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6192

Create unit test to automatically compare MR related classes and mapred-default.xml

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Create a unit test that will automatically compare the fields in the various MapReduce related classes and mapred-default.xml. It should throw an error if a property is missing in either the class or the file.

      1. MAPREDUCE-6192.001.patch
        3 kB
        Ray Chiang
      2. MAPREDUCE-6192.002.patch
        4 kB
        Ray Chiang
      3. MAPREDUCE-6192.003.patch
        5 kB
        Ray Chiang
      4. MAPREDUCE-6192.004.patch
        5 kB
        Ray Chiang
      5. MAPREDUCE-6192.005.patch
        29 kB
        Ray Chiang
      6. MAPREDUCE-6192.006.patch
        8 kB
        Ray Chiang
      7. MAPREDUCE-6192.007.patch
        8 kB
        Ray Chiang
      8. MAPREDUCE-6192.branch-2.007.patch
        9 kB
        Ray Chiang

        Issue Links

          Activity

          Hide
          qwertymaniac Harsh J added a comment -

          Some properties may intentionally be skipped from presence in mapred-default.xml BTW.

          Show
          qwertymaniac Harsh J added a comment - Some properties may intentionally be skipped from presence in mapred-default.xml BTW.
          Hide
          rchiang Ray Chiang added a comment -

          That's fine. The code allows for that, but it would be nice to have an intentional list documented somewhere rather than an unknown quantity of missing properties.

          Show
          rchiang Ray Chiang added a comment - That's fine. The code allows for that, but it would be nice to have an intentional list documented somewhere rather than an unknown quantity of missing properties.
          Hide
          rchiang Ray Chiang added a comment -

          Upload initial unit test.

          Show
          rchiang Ray Chiang added a comment - Upload initial unit test.
          Hide
          rkanter Robert Kanter added a comment -

          +1 pending Jenkins (which seems to be down at the moment)

          Show
          rkanter Robert Kanter added a comment - +1 pending Jenkins (which seems to be down at the moment)
          Hide
          rchiang Ray Chiang added a comment -

          My bad. I didn't click on "Submit Patch". I expect the unit test to fail until MAPREDUCE-6057 is committed first.

          Show
          rchiang Ray Chiang added a comment - My bad. I didn't click on "Submit Patch". I expect the unit test to fail until MAPREDUCE-6057 is committed first.
          Hide
          rchiang Ray Chiang added a comment -
          • Update base regex for valid properties
          • Does not currently error out on mismatches. This doesn't need to be turned on until the blocking JIRAs are fixed.
          Show
          rchiang Ray Chiang added a comment - Update base regex for valid properties Does not currently error out on mismatches. This doesn't need to be turned on until the blocking JIRAs are fixed.
          Hide
          rchiang Ray Chiang added a comment -

          Submit for testing

          Show
          rchiang Ray Chiang added a comment - Submit for testing
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12693507/MAPREDUCE-6192.002.patch
          against trunk revision a122278.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5115//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5115//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12693507/MAPREDUCE-6192.002.patch against trunk revision a122278. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5115//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5115//console This message is automatically generated.
          Hide
          rchiang Ray Chiang added a comment -

          RE: Failing unit tests

          Both tests pass in my tree.

          Show
          rchiang Ray Chiang added a comment - RE: Failing unit tests Both tests pass in my tree.
          Hide
          rchiang Ray Chiang added a comment -

          Update with new property to ignore

          Show
          rchiang Ray Chiang added a comment - Update with new property to ignore
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12702984/MAPREDUCE-6192.003.patch
          against trunk revision 952640f.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The following test timeouts occurred in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:

          org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5243//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5243//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12702984/MAPREDUCE-6192.003.patch against trunk revision 952640f. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The following test timeouts occurred in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5243//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5243//console This message is automatically generated.
          Hide
          rchiang Ray Chiang added a comment -

          Add FileOutputCommitter class for new property

          Show
          rchiang Ray Chiang added a comment - Add FileOutputCommitter class for new property
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12705657/MAPREDUCE-6192.004.patch
          against trunk revision 61a4c7f.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5313//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5313//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705657/MAPREDUCE-6192.004.patch against trunk revision 61a4c7f. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5313//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5313//console This message is automatically generated.
          Hide
          rchiang Ray Chiang added a comment -
          • Still have to add description to properties.
          • Turn on all real error checking.
          Show
          rchiang Ray Chiang added a comment - Still have to add description to properties. Turn on all real error checking.
          Hide
          hadoopqa Hadoop QA added a comment -

          The patch artifact directory on has been removed!
          This is a fatal error for test-patch.sh. Aborting.
          Jenkins (node H8) information at https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5454/ may provide some hints.

          Show
          hadoopqa Hadoop QA added a comment - The patch artifact directory on has been removed! This is a fatal error for test-patch.sh. Aborting. Jenkins (node H8) information at https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5454/ may provide some hints.
          Hide
          rchiang Ray Chiang added a comment -

          Previous patch contained fixes for the next JIRA. Update patch with version that only compares one-way.

          Show
          rchiang Ray Chiang added a comment - Previous patch contained fixes for the next JIRA. Update patch with version that only compares one-way.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 5m 9s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          -1 javac 7m 25s The applied patch generated 126 additional warning messages.
          +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 3m 52s There were no new checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          +1 findbugs 2m 34s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 25m 30s Tests passed in hadoop-common.
          +1 mapreduce tests 14m 26s Tests passed in hadoop-mapreduce-client-app.
              61m 22s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12728401/MAPREDUCE-6192.006.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 5e67c4d
          javac https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/artifact/patchprocess/diffJavacWarnings.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 9s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. -1 javac 7m 25s The applied patch generated 126 additional warning messages. +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 3m 52s There were no new checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. +1 findbugs 2m 34s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 25m 30s Tests passed in hadoop-common. +1 mapreduce tests 14m 26s Tests passed in hadoop-mapreduce-client-app.     61m 22s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12728401/MAPREDUCE-6192.006.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 5e67c4d javac https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/artifact/patchprocess/diffJavacWarnings.txt hadoop-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/artifact/patchprocess/testrun_hadoop-common.txt hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5456/console This message was automatically generated.
          Hide
          rchiang Ray Chiang added a comment -

          Ironically, my test generates the deprecation warnings for properties that are still documented in mapred-default.xml.

          Show
          rchiang Ray Chiang added a comment - Ironically, my test generates the deprecation warnings for properties that are still documented in mapred-default.xml.
          Hide
          rkanter Robert Kanter added a comment -

          To "fix" the deprecation warnings, can you add the @SuppressWarnings("deprecation") thing where appropriate?

          Show
          rkanter Robert Kanter added a comment - To "fix" the deprecation warnings, can you add the @SuppressWarnings("deprecation") thing where appropriate?
          Hide
          rchiang Ray Chiang added a comment -

          Will do.

          Show
          rchiang Ray Chiang added a comment - Will do.
          Hide
          rchiang Ray Chiang added a comment -

          Will do.

          Show
          rchiang Ray Chiang added a comment - Will do.
          Hide
          rchiang Ray Chiang added a comment -

          Add @SuppressWarnings to test method.

          Show
          rchiang Ray Chiang added a comment - Add @SuppressWarnings to test method.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 5m 9s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 28s There were no new javac warning messages.
          +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 5m 24s There were no new checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 2m 34s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 common tests 23m 24s Tests passed in hadoop-common.
          +1 mapreduce tests 9m 9s Tests passed in hadoop-mapreduce-client-app.
              55m 35s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12729318/MAPREDUCE-6192.007.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 4c1af15
          hadoop-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 9s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 28s There were no new javac warning messages. +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 5m 24s There were no new checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 34s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 24s Tests passed in hadoop-common. +1 mapreduce tests 9m 9s Tests passed in hadoop-mapreduce-client-app.     55m 35s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12729318/MAPREDUCE-6192.007.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 4c1af15 hadoop-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/artifact/patchprocess/testrun_hadoop-common.txt hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5475/console This message was automatically generated.
          Hide
          rkanter Robert Kanter added a comment -

          LGTM +1

          Show
          rkanter Robert Kanter added a comment - LGTM +1
          Hide
          rkanter Robert Kanter added a comment -

          Ray Chiang, it works fine on trunk, but it looks like there's some differences in branch-2. Can you look into this?

          Running org.apache.hadoop.mapreduce.TestMapreduceConfigFields
          Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.624 sec <<< FAILURE! - in org.apache.hadoop.mapreduce.TestMapreduceConfigFields
          testCompareXmlAgainstConfigurationClass(org.apache.hadoop.mapreduce.TestMapreduceConfigFields)  Time elapsed: 0.497 sec  <<< FAILURE!
          java.lang.AssertionError: mapred-default.xml has 4 properties missing in  interface org.apache.hadoop.mapreduce.MRJobConfig  interface org.apache.hadoop.mapreduce.MRConfig  class org.apache.hadoop.mapreduce.v2.jobhistory.JHAdminConfig  class org.apache.hadoop.mapred.ShuffleHandler  class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat  class org.apache.hadoop.mapreduce.lib.input.FileInputFormat  class org.apache.hadoop.mapreduce.Job  class org.apache.hadoop.mapreduce.lib.input.NLineInputFormat  class org.apache.hadoop.mapred.JobConf  class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.junit.Assert.assertTrue(Assert.java:41)
          	at org.apache.hadoop.conf.TestConfigurationFieldsBase.testCompareXmlAgainstConfigurationClass(TestConfigurationFieldsBase.java:468)
                  ...
          
          Show
          rkanter Robert Kanter added a comment - Ray Chiang , it works fine on trunk, but it looks like there's some differences in branch-2. Can you look into this? Running org.apache.hadoop.mapreduce.TestMapreduceConfigFields Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.624 sec <<< FAILURE! - in org.apache.hadoop.mapreduce.TestMapreduceConfigFields testCompareXmlAgainstConfigurationClass(org.apache.hadoop.mapreduce.TestMapreduceConfigFields) Time elapsed: 0.497 sec <<< FAILURE! java.lang.AssertionError: mapred-default.xml has 4 properties missing in interface org.apache.hadoop.mapreduce.MRJobConfig interface org.apache.hadoop.mapreduce.MRConfig class org.apache.hadoop.mapreduce.v2.jobhistory.JHAdminConfig class org.apache.hadoop.mapred.ShuffleHandler class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat class org.apache.hadoop.mapreduce.lib.input.FileInputFormat class org.apache.hadoop.mapreduce.Job class org.apache.hadoop.mapreduce.lib.input.NLineInputFormat class org.apache.hadoop.mapred.JobConf class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.hadoop.conf.TestConfigurationFieldsBase.testCompareXmlAgainstConfigurationClass(TestConfigurationFieldsBase.java:468) ...
          Hide
          rchiang Ray Chiang added a comment -

          MAPREDUCE-6057 (Remove obsolete entries from mapred-default.xml) was committed to trunk, but not to branch-2. Those obsolete entries need a waiver in branch-2 version of this patch.

          Show
          rchiang Ray Chiang added a comment - MAPREDUCE-6057 (Remove obsolete entries from mapred-default.xml) was committed to trunk, but not to branch-2. Those obsolete entries need a waiver in branch-2 version of this patch.
          Hide
          rchiang Ray Chiang added a comment -

          I've put up a separate branch-2 patch. It makes the TestMapreduceConfigFields equivalent in branch-2 and trunk.

          Show
          rchiang Ray Chiang added a comment - I've put up a separate branch-2 patch. It makes the TestMapreduceConfigFields equivalent in branch-2 and trunk.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 5m 6s Pre-patch branch-2 compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          -1 patch 0m 54s The patch command could not apply the patch.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12730361/MAPREDUCE-6192.branch-2.007.patch
          Optional Tests javac unit findbugs checkstyle
          git revision branch-2 / aefadb8
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5643/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 6s Pre-patch branch-2 compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. -1 patch 0m 54s The patch command could not apply the patch. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12730361/MAPREDUCE-6192.branch-2.007.patch Optional Tests javac unit findbugs checkstyle git revision branch-2 / aefadb8 Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5643/console This message was automatically generated.
          Hide
          rkanter Robert Kanter added a comment -

          +1

          Show
          rkanter Robert Kanter added a comment - +1
          Hide
          rkanter Robert Kanter added a comment -

          Thanks Ray. Committed to trunk and branch-2!

          Show
          rkanter Robert Kanter added a comment - Thanks Ray. Committed to trunk and branch-2!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7741 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7741/)
          MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7741 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7741/ ) MAPREDUCE-6192 . Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f) hadoop-mapreduce-project/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/186/)
          MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f)

          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/186/ ) MAPREDUCE-6192 . Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #919 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/919/)
          MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #919 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/919/ ) MAPREDUCE-6192 . Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f) hadoop-mapreduce-project/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2117 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2117/)
          MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2117 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2117/ ) MAPREDUCE-6192 . Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #176 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/176/)
          MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #176 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/176/ ) MAPREDUCE-6192 . Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f) hadoop-mapreduce-project/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2135 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2135/)
          MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2135 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2135/ ) MAPREDUCE-6192 . Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java hadoop-mapreduce-project/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/186/)
          MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #186 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/186/ ) MAPREDUCE-6192 . Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) (rkanter: rev 9809a16d3c8068beccbf0106e99c7ede6ba11e0f) hadoop-mapreduce-project/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfigurationFieldsBase.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestMapreduceConfigFields.java
          Hide
          rchiang Ray Chiang added a comment -

          Thanks for the review and the commit!

          Show
          rchiang Ray Chiang added a comment - Thanks for the review and the commit!

            People

            • Assignee:
              rchiang Ray Chiang
              Reporter:
              rchiang Ray Chiang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development