Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6954

Disable erasure coding for files that are uploaded to the MR staging area

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha4
    • Fix Version/s: 3.0.0-beta1
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Depending on the encoder/decoder used and the type or MR workload, EC might negatively affect the performance of an MR job if too many files are localized.

      In such a scenario, users might want to disable EC in the staging area to speed up the execution.

      1. MAPREDUCE-6954-001.patch
        4 kB
        Peter Bacsko
      2. MAPREDUCE-6954-002.patch
        8 kB
        Peter Bacsko
      3. MAPREDUCE-6954-003.patch
        8 kB
        Peter Bacsko
      4. MAPREDUCE-6954-004.patch
        8 kB
        Peter Bacsko

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12898 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12898/)
          MAPREDUCE-6954. Disable erasure coding for files that are uploaded to (rkanter: rev 0adc0471d0c06f66a31060f270dcb50a7b4ffafa)

          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobResourceUploader.java
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12898 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12898/ ) MAPREDUCE-6954 . Disable erasure coding for files that are uploaded to (rkanter: rev 0adc0471d0c06f66a31060f270dcb50a7b4ffafa) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobResourceUploader.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
          Hide
          pbacsko Peter Bacsko added a comment -

          Thanks for fixing it.

          Show
          pbacsko Peter Bacsko added a comment - Thanks for fixing it.
          Show
          rkanter Robert Kanter added a comment - Oops, I was committing some other stuff that day too, and accidentally put the name of one of those in the commit messages trunk: https://github.com/apache/hadoop/commit/3a8d57a0a2e047b34be82f602a2b6cf5593d2125 branch-3.0: https://github.com/apache/hadoop/commit/293ab432e116b93ada6ee56e518ae07aa853ab41 I've just reverted those two: trunk: https://github.com/apache/hadoop/commit/5f496683fb00ba26a6bf5a506ae87d4bc4088727 https://github.com/apache/hadoop/commit/3847b8c861205d9c85c866ec3477e8b0d19bd659 And recommitted the code with the correct message: trunk: https://github.com/apache/hadoop/commit/0adc0471d0c06f66a31060f270dcb50a7b4ffafa https://github.com/apache/hadoop/commit/917e875e511589c297971f6a14fd85de7925409f
          Hide
          pbacsko Peter Bacsko added a comment -

          Robert Kanter something is not quite right. I can't see this change on any of the branches you mentioned.

          Show
          pbacsko Peter Bacsko added a comment - Robert Kanter something is not quite right. I can't see this change on any of the branches you mentioned.
          Hide
          rkanter Robert Kanter added a comment -

          Thanks Peter Bacsko. Committed to trunk and branch-3.0!

          Show
          rkanter Robert Kanter added a comment - Thanks Peter Bacsko . Committed to trunk and branch-3.0!
          Hide
          rkanter Robert Kanter added a comment -

          +1

          Show
          rkanter Robert Kanter added a comment - +1
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 12m 40s trunk passed
          +1 compile 0m 25s trunk passed
          +1 checkstyle 0m 23s trunk passed
          +1 mvnsite 0m 27s trunk passed
          +1 findbugs 0m 47s trunk passed
          +1 javadoc 0m 21s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 21s the patch passed
          +1 javac 0m 21s the patch passed
          +1 checkstyle 0m 19s the patch passed
          +1 mvnsite 0m 25s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 0m 51s the patch passed
          +1 javadoc 0m 18s the patch passed
                Other Tests
          +1 unit 2m 44s hadoop-mapreduce-client-core in the patch passed.
          +1 asflicense 0m 13s The patch does not generate ASF License warnings.
          21m 34s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue MAPREDUCE-6954
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887325/MAPREDUCE-6954-004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 8a7126aa2100 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 50764ef
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7138/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7138/console
          Powered by Apache Yetus 0.5.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 12m 40s trunk passed +1 compile 0m 25s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 27s trunk passed +1 findbugs 0m 47s trunk passed +1 javadoc 0m 21s trunk passed       Patch Compile Tests +1 mvninstall 0m 23s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed +1 checkstyle 0m 19s the patch passed +1 mvnsite 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 0m 51s the patch passed +1 javadoc 0m 18s the patch passed       Other Tests +1 unit 2m 44s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 13s The patch does not generate ASF License warnings. 21m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue MAPREDUCE-6954 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887325/MAPREDUCE-6954-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 8a7126aa2100 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 50764ef Default Java 1.8.0_144 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7138/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7138/console Powered by Apache Yetus 0.5.0 http://yetus.apache.org This message was automatically generated.
          Hide
          pbacsko Peter Bacsko added a comment -

          I had to upload a new patch, because I forgot to modify mapred-default.xml

          Show
          pbacsko Peter Bacsko added a comment - I had to upload a new patch, because I forgot to modify mapred-default.xml
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 17m 9s trunk passed
          +1 compile 0m 36s trunk passed
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 0m 42s trunk passed
          +1 findbugs 1m 7s trunk passed
          +1 javadoc 0m 30s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 32s the patch passed
          +1 compile 0m 28s the patch passed
          +1 javac 0m 28s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 0m 32s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 findbugs 1m 10s the patch passed
          +1 javadoc 0m 23s the patch passed
                Other Tests
          +1 unit 3m 7s hadoop-mapreduce-client-core in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          28m 49s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue MAPREDUCE-6954
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887316/MAPREDUCE-6954-003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 6d897802dcb4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 50764ef
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7137/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7137/console
          Powered by Apache Yetus 0.5.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 17m 9s trunk passed +1 compile 0m 36s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 0m 42s trunk passed +1 findbugs 1m 7s trunk passed +1 javadoc 0m 30s trunk passed       Patch Compile Tests +1 mvninstall 0m 32s the patch passed +1 compile 0m 28s the patch passed +1 javac 0m 28s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 32s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 1m 10s the patch passed +1 javadoc 0m 23s the patch passed       Other Tests +1 unit 3m 7s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 28m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue MAPREDUCE-6954 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887316/MAPREDUCE-6954-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 6d897802dcb4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 50764ef Default Java 1.8.0_144 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7137/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7137/console Powered by Apache Yetus 0.5.0 http://yetus.apache.org This message was automatically generated.
          Hide
          pbacsko Peter Bacsko added a comment -

          I changed the boolean parameter as you requested. I agree it was confusing.
          The default value of the property is now "false" instead of "true", that is it will try to disable EC.

          Show
          pbacsko Peter Bacsko added a comment - I changed the boolean parameter as you requested. I agree it was confusing. The default value of the property is now "false" instead of "true", that is it will try to disable EC.
          Hide
          rkanter Robert Kanter added a comment -

          Almost there:

          1. I'm not a fan of double negatives in booleans - they make things hard to read. I understand what you're doing here, relying on the default value to be true instead of setting it (that's a good idea btw), but instead of using !shouldBeDisabled let's use default instead. That's much easier to follow.
            • In a similar vein, let's rename testErasureCodingNotDisabled to testErasureCodingDefault.
          2. There's some new checkstyle warnings about indenting and line length.
          Show
          rkanter Robert Kanter added a comment - Almost there: I'm not a fan of double negatives in booleans - they make things hard to read. I understand what you're doing here, relying on the default value to be true instead of setting it (that's a good idea btw), but instead of using !shouldBeDisabled let's use default instead. That's much easier to follow. In a similar vein, let's rename testErasureCodingNotDisabled to testErasureCodingDefault . There's some new checkstyle warnings about indenting and line length.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                trunk Compile Tests
          +1 mvninstall 13m 25s trunk passed
          +1 compile 0m 26s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 28s trunk passed
          +1 findbugs 0m 46s trunk passed
          +1 javadoc 0m 22s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 21s the patch passed
          +1 javac 0m 21s the patch passed
          -1 checkstyle 0m 22s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 3 new + 538 unchanged - 0 fixed = 541 total (was 538)
          +1 mvnsite 0m 24s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          +1 findbugs 0m 49s the patch passed
          +1 javadoc 0m 19s the patch passed
                Other Tests
          +1 unit 2m 42s hadoop-mapreduce-client-core in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          22m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue MAPREDUCE-6954
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887129/MAPREDUCE-6954-002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 236c63233a8f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / b9465bb
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7134/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7134/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7134/console
          Powered by Apache Yetus 0.5.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 13m 25s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 28s trunk passed +1 findbugs 0m 46s trunk passed +1 javadoc 0m 22s trunk passed       Patch Compile Tests +1 mvninstall 0m 23s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed -1 checkstyle 0m 22s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 3 new + 538 unchanged - 0 fixed = 541 total (was 538) +1 mvnsite 0m 24s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 0m 49s the patch passed +1 javadoc 0m 19s the patch passed       Other Tests +1 unit 2m 42s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 22m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue MAPREDUCE-6954 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12887129/MAPREDUCE-6954-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 236c63233a8f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b9465bb Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7134/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7134/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7134/console Powered by Apache Yetus 0.5.0 http://yetus.apache.org This message was automatically generated.
          Hide
          rkanter Robert Kanter added a comment -

          Overall looks good. A couple minor things:

          • Update mapred-default like you suggested
          • The staging dir config is yarn.app.mapreduce.am.staging-dir and defined in MRJobConfig as MR_AM_STAGING_DIR = MR_AM_PREFIX+"staging-dir". I think we should be consistent with this and make MR_JOB_STAGING_ERASURECODING_ENABLED = "mapreduce.job.staging-dir.erasurecoding.enabled" be MR_AM_STAGING_DIR_ERASURECODING_ENABLED = MR_AM_STAGING_DIR + "erasurecoding.enabled" which would resolve to yarn.app.mapreduce.am.staging-dir.erasurecoding.enabled.
          • It would be good to add a unit test.
          Show
          rkanter Robert Kanter added a comment - Overall looks good. A couple minor things: Update mapred-default like you suggested The staging dir config is yarn.app.mapreduce.am.staging-dir and defined in MRJobConfig as MR_AM_STAGING_DIR = MR_AM_PREFIX+"staging-dir" . I think we should be consistent with this and make MR_JOB_STAGING_ERASURECODING_ENABLED = "mapreduce.job.staging-dir.erasurecoding.enabled" be MR_AM_STAGING_DIR_ERASURECODING_ENABLED = MR_AM_STAGING_DIR + "erasurecoding.enabled" which would resolve to yarn.app.mapreduce.am.staging-dir.erasurecoding.enabled . It would be good to add a unit test.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 26s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          +1 mvninstall 17m 47s trunk passed
          +1 compile 0m 36s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 0m 37s trunk passed
          +1 findbugs 1m 2s trunk passed
          +1 javadoc 0m 24s trunk passed
                Patch Compile Tests
          +1 mvninstall 0m 29s the patch passed
          +1 compile 0m 23s the patch passed
          +1 javac 0m 23s the patch passed
          -1 checkstyle 0m 21s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 2 new + 530 unchanged - 0 fixed = 532 total (was 530)
          +1 mvnsite 0m 27s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 0m 56s the patch passed
          +1 javadoc 0m 19s the patch passed
                Other Tests
          +1 unit 2m 55s hadoop-mapreduce-client-core in the patch passed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          28m 16s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:71bbb86
          JIRA Issue MAPREDUCE-6954
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886071/MAPREDUCE-6954-001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle
          uname Linux 122d57defd22 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 5bbca80
          Default Java 1.8.0_144
          findbugs v3.1.0-RC1
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7124/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7124/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7124/console
          Powered by Apache Yetus 0.5.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests +1 mvninstall 17m 47s trunk passed +1 compile 0m 36s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 0m 37s trunk passed +1 findbugs 1m 2s trunk passed +1 javadoc 0m 24s trunk passed       Patch Compile Tests +1 mvninstall 0m 29s the patch passed +1 compile 0m 23s the patch passed +1 javac 0m 23s the patch passed -1 checkstyle 0m 21s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core: The patch generated 2 new + 530 unchanged - 0 fixed = 532 total (was 530) +1 mvnsite 0m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 0m 56s the patch passed +1 javadoc 0m 19s the patch passed       Other Tests +1 unit 2m 55s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 28m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue MAPREDUCE-6954 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12886071/MAPREDUCE-6954-001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle uname Linux 122d57defd22 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5bbca80 Default Java 1.8.0_144 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7124/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7124/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7124/console Powered by Apache Yetus 0.5.0 http://yetus.apache.org This message was automatically generated.
          Hide
          pbacsko Peter Bacsko added a comment -

          Forgot to update mapred-default.xml, will do it in the next patch.

          Show
          pbacsko Peter Bacsko added a comment - Forgot to update mapred-default.xml , will do it in the next patch.

            People

            • Assignee:
              pbacsko Peter Bacsko
              Reporter:
              pbacsko Peter Bacsko
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development