Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5870

Support for passing Job priority through Application Submission Context in Mapreduce Side

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Job Priority support in MapReduce.

      1. Job Priority can be set from client side as below (Configuration and api).

      • JobConf.getJobPriority() and JobConf.setJobPriority(JobPriority priority)
      • We can also use configuration "mapreduce.job.priority".

      Now this Job priority can be passed in Application Submission context from Client side.
      Here we can reuse the MRJobConfig.PRIORITY configuration.
      2. CLI changes to support --set-priority. Run time change of JobPriority is also to be handled.
      3. Change Job to have the support for setPriority and getPriority (getter is handled with JobStatus)

      1. 0001-MAPREDUCE-5870.patch
        3 kB
        Sunil G
      2. 0002-MAPREDUCE-5870.patch
        6 kB
        Sunil G
      3. 0003-MAPREDUCE-5870.patch
        6 kB
        Sunil G
      4. 0004-MAPREDUCE-5870.patch
        13 kB
        Sunil G
      5. 0005-MAPREDUCE-5870.patch
        32 kB
        Sunil G
      6. 0006-MAPREDUCE-5870.patch
        32 kB
        Sunil G
      7. 0007-MAPREDUCE-5870.patch
        29 kB
        Sunil G
      8. 0008-MAPREDUCE-5870.patch
        33 kB
        Sunil G
      9. 0009-MAPREDUCE-5870.patch
        33 kB
        Sunil G
      10. 0010-MAPREDUCE-5870.patch
        34 kB
        Sunil G
      11. 0011-MAPREDUCE-5870.patch
        36 kB
        Sunil G
      12. 0012-MAPREDUCE-5870.patch
        36 kB
        Sunil G
      13. Yarn-2002.1.patch
        2 kB
        Sunil G

        Issue Links

          Activity

          Hide
          sunilg Sunil G added a comment -

          Test cases are not added here, intended to cover as part of Yarn-2004

          Show
          sunilg Sunil G added a comment - Test cases are not added here, intended to cover as part of Yarn-2004
          Hide
          jlowe Jason Lowe added a comment -

          Moving this to MAPREDUCE since that's where the changes need to be made. Will link this issue back to YARN-1963.

          I think a small unit test should be added as part of this change to verify that when a priority is set the resulting submission context from YARNRunner has the appropriate priority setting. I suspect the tests in YARN-2004 will be more of an integration test rather than a unit test.

          Show
          jlowe Jason Lowe added a comment - Moving this to MAPREDUCE since that's where the changes need to be made. Will link this issue back to YARN-1963 . I think a small unit test should be added as part of this change to verify that when a priority is set the resulting submission context from YARNRunner has the appropriate priority setting. I suspect the tests in YARN-2004 will be more of an integration test rather than a unit test.
          Hide
          devaraj.k Devaraj K added a comment -

          Please refer TestTypeConverter.java for adding tests for this.

          Show
          devaraj.k Devaraj K added a comment - Please refer TestTypeConverter.java for adding tests for this.
          Hide
          sunilg Sunil G added a comment -

          Thank you Jason and Devaraj for the comments. I will upload the patch with test cases as mentioned.
          Also in the parent jira, there are some discussions happening on same. So based on that, the patch will be uploaded.

          Show
          sunilg Sunil G added a comment - Thank you Jason and Devaraj for the comments. I will upload the patch with test cases as mentioned. Also in the parent jira, there are some discussions happening on same. So based on that, the patch will be uploaded.
          Hide
          sunilg Sunil G added a comment -

          Uploading an updated version of patch with test case.

          In YARN, if a client is not setting application priority in its submission context, then default priority from queue is considered.

          Hence in MR, I did not set any default priority if user is not specifying priority in mapreduce.job.priority. Please share thoughts.

          Show
          sunilg Sunil G added a comment - Uploading an updated version of patch with test case. In YARN, if a client is not setting application priority in its submission context, then default priority from queue is considered. Hence in MR, I did not set any default priority if user is not specifying priority in mapreduce.job.priority . Please share thoughts.
          Hide
          sunilg Sunil G added a comment -

          Hi Jason Lowe Devaraj K
          Could you please take a look on this patch. YARN-2003 is committed in trunk, so with this change we can submit an MR job with app priority.

          Show
          sunilg Sunil G added a comment - Hi Jason Lowe Devaraj K Could you please take a look on this patch. YARN-2003 is committed in trunk, so with this change we can submit an MR job with app priority.
          Hide
          jianhe Jian He added a comment -

          patch looks good to me, triggering jenkins

          Show
          jianhe Jian He added a comment - patch looks good to me, triggering jenkins
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 15m 41s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 44s There were no new javac warning messages.
          +1 javadoc 9m 40s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 33s The applied patch generated 1 new checkstyle issues (total was 28, now 28).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 20s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 mapreduce tests 105m 41s Tests failed in hadoop-mapreduce-client-jobclient.
              142m 27s  



          Reason Tests
          Failed unit tests hadoop.mapreduce.TestMRJobClient



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12745043/0001-MAPREDUCE-5870.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fbd6063
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-jobclient.txt
          hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 41s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 33s The applied patch generated 1 new checkstyle issues (total was 28, now 28). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 20s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 mapreduce tests 105m 41s Tests failed in hadoop-mapreduce-client-jobclient.     142m 27s   Reason Tests Failed unit tests hadoop.mapreduce.TestMRJobClient Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12745043/0001-MAPREDUCE-5870.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fbd6063 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-jobclient.txt hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5910/console This message was automatically generated.
          Hide
          jianhe Jian He added a comment -

          Sunil G, could you check if the test failure is related ?

          Show
          jianhe Jian He added a comment - Sunil G , could you check if the test failure is related ?
          Hide
          sunilg Sunil G added a comment -

          Thank you Jian He for review and sharing the comments.

          I have looked into the test failures and analyzed the problem. It seems we still have implementat

          Show
          sunilg Sunil G added a comment - Thank you Jian He for review and sharing the comments. I have looked into the test failures and analyzed the problem. It seems we still have implementat
          Hide
          sunilg Sunil G added a comment -

          Thank you Jian He for review and sharing the comments.
          I have looked into the test failures and analyzed the problem. It seems we still have implementation of old MRV1 priority in code.

          @InterfaceAudience.Public
          @InterfaceStability.Stable
          public enum JobPriority {
          
            VERY_HIGH,
            HIGH,
            NORMAL,
            LOW,
            VERY_LOW;
            
          }
          

          This enum is used in many places in code. Also we have some stable apis in JobConf related to JobPriority.

           public void setJobPriority(JobPriority prio)
           public JobPriority getJobPriority()
          

          Do we still need to keep any backward compatibility support to this? If so, we can do a type convert for this macro and set priority accordingly in YarnRunner. Otherwise we may need to remove some of unused test cases from MR related to old way of setting priority.
          Kindly advice.

          Show
          sunilg Sunil G added a comment - Thank you Jian He for review and sharing the comments. I have looked into the test failures and analyzed the problem. It seems we still have implementation of old MRV1 priority in code. @InterfaceAudience.Public @InterfaceStability.Stable public enum JobPriority { VERY_HIGH, HIGH, NORMAL, LOW, VERY_LOW; } This enum is used in many places in code. Also we have some stable apis in JobConf related to JobPriority. public void setJobPriority(JobPriority prio) public JobPriority getJobPriority() Do we still need to keep any backward compatibility support to this? If so, we can do a type convert for this macro and set priority accordingly in YarnRunner. Otherwise we may need to remove some of unused test cases from MR related to old way of setting priority. Kindly advice.
          Hide
          jianhe Jian He added a comment -

          Sunil G, thanks for the investigation.
          Given that these implementation already exist in MR, I would prefer continue supporting this.

          Show
          jianhe Jian He added a comment - Sunil G , thanks for the investigation. Given that these implementation already exist in MR, I would prefer continue supporting this.
          Hide
          sunilg Sunil G added a comment -

          Thank you Jian He for the clarification.
          Uploading a patch based on same.

          Show
          sunilg Sunil G added a comment - Thank you Jian He for the clarification. Uploading a patch based on same.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 39s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 44s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 55s The applied patch generated 11 new checkstyle issues (total was 33, now 44).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 24s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 0s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 0m 47s Tests passed in hadoop-mapreduce-client-common.
          +1 mapreduce tests 110m 36s Tests passed in hadoop-mapreduce-client-jobclient.
              150m 54s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747222/0002-MAPREDUCE-5870.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 156f24e
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-common.txt
          hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
          hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 39s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 44s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 55s The applied patch generated 11 new checkstyle issues (total was 33, now 44). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 24s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 0s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 0m 47s Tests passed in hadoop-mapreduce-client-common. +1 mapreduce tests 110m 36s Tests passed in hadoop-mapreduce-client-jobclient.     150m 54s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747222/0002-MAPREDUCE-5870.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 156f24e checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-common.txt hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5912/console This message was automatically generated.
          Hide
          jianhe Jian He added a comment -

          Sunil G, thanks for updating,

          • should we use the JobConf.getJobPriority API so that it can accept the current CLI specified priority too ?
                String jobPriority = jobConf.get(MRJobConfig.PRIORITY);
            
          • to simplify a little bit,
                    int iPriority = TypeConverter.toYarn(jobPriority);
                    // If the given input not a JobPriority enum, verify whether its an
                    // integer.
                    if (0 == iPriority) {
                      iPriority = Integer.parseInt(jobPriority);
                    }
            

            we can do something like

                    try {
                      iPriority = TypeConverter.toYarn(jobPriority);
                    } catch (IllegalArgumentException exception) {
                      iPriority = Integer.parseInt(jobPriority);
                    }
            
          Show
          jianhe Jian He added a comment - Sunil G , thanks for updating, should we use the JobConf.getJobPriority API so that it can accept the current CLI specified priority too ? String jobPriority = jobConf.get(MRJobConfig.PRIORITY); to simplify a little bit, int iPriority = TypeConverter.toYarn(jobPriority); // If the given input not a JobPriority enum , verify whether its an // integer. if (0 == iPriority) { iPriority = Integer .parseInt(jobPriority); } we can do something like try { iPriority = TypeConverter.toYarn(jobPriority); } catch (IllegalArgumentException exception) { iPriority = Integer .parseInt(jobPriority); }
          Hide
          sunilg Sunil G added a comment -

          Thank you Jian He for the comments. Uploading a patch addressing the same. If user input is neither enum nor an integer, createApplicationSubmissionContext will throw NumberFormatExcpetion, I hope this is fine (avoided adding one more try block around parseInt. Pls share your thoughts.

          Show
          sunilg Sunil G added a comment - Thank you Jian He for the comments. Uploading a patch addressing the same. If user input is neither enum nor an integer, createApplicationSubmissionContext will throw NumberFormatExcpetion, I hope this is fine (avoided adding one more try block around parseInt . Pls share your thoughts.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 35s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 36s There were no new javac warning messages.
          +1 javadoc 9m 37s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 55s The applied patch generated 1 new checkstyle issues (total was 33, now 34).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 0m 46s Tests passed in hadoop-mapreduce-client-common.
          +1 mapreduce tests 109m 10s Tests passed in hadoop-mapreduce-client-jobclient.
              149m 2s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12747341/0003-MAPREDUCE-5870.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 2196e39
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-common.txt
          hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
          hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 35s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 36s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 55s The applied patch generated 1 new checkstyle issues (total was 33, now 34). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 0m 46s Tests passed in hadoop-mapreduce-client-common. +1 mapreduce tests 109m 10s Tests passed in hadoop-mapreduce-client-jobclient.     149m 2s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747341/0003-MAPREDUCE-5870.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2196e39 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-common.txt hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5913/console This message was automatically generated.
          Hide
          eepayne Eric Payne added a comment -

          Sunil G, for what it's worth, I have downloaded the latest patch (version 003) and tested and verified it in conjunction with the changes that were made for YARN-2003.

          I performed the following sleep jobs with 10 tasks each. My one-node cluster can run 5 containers at once.

          • I submit sleep job1 to the default queue, setting -Dmapreduce.job.priority=LOW
          • Job1 starts running 5 containers and has 5 tasks pending.
          • I submit sleep job2 to the default queue, setting -Dmapreduce.job.priority=HIGH
          • All 10 job2 tasks are pending.
          • Once tasks from job1 complete, job2 gets the containers. Although job1 has 5 tasks pending, the number of running tasks for job1 remains 0 until job2 has no more pending tasks and job2's running tasks begin to complete.
          • At that point, job1's tasks begin again to receive containers.

          I also verified that you can specify -Dmapreduce.job.priority=number, and the container allocations go to the higher numbered jobs.
          Finally, I verified that if you make the priority higher than the cluster max, it silently sets the job priority to cluster max.

          So, the bottom line is LGTM
          +1

          Show
          eepayne Eric Payne added a comment - Sunil G , for what it's worth, I have downloaded the latest patch (version 003) and tested and verified it in conjunction with the changes that were made for YARN-2003 . I performed the following sleep jobs with 10 tasks each. My one-node cluster can run 5 containers at once. I submit sleep job1 to the default queue, setting -Dmapreduce.job.priority=LOW Job1 starts running 5 containers and has 5 tasks pending. I submit sleep job2 to the default queue, setting -Dmapreduce.job.priority=HIGH All 10 job2 tasks are pending. Once tasks from job1 complete, job2 gets the containers. Although job1 has 5 tasks pending, the number of running tasks for job1 remains 0 until job2 has no more pending tasks and job2's running tasks begin to complete. At that point, job1's tasks begin again to receive containers. I also verified that you can specify -Dmapreduce.job.priority= number , and the container allocations go to the higher numbered jobs. Finally, I verified that if you make the priority higher than the cluster max, it silently sets the job priority to cluster max. So, the bottom line is LGTM +1
          Hide
          eepayne Eric Payne added a comment -

          Hi Sunil G,
          One thing I forgot to mention was that I had to add the following line to YARNRunner.java to get it to compile.

          import org.apache.hadoop.yarn.api.records.Priority;
          
          Show
          eepayne Eric Payne added a comment - Hi Sunil G , One thing I forgot to mention was that I had to add the following line to YARNRunner.java to get it to compile. import org.apache.hadoop.yarn.api.records.Priority;
          Hide
          sunilg Sunil G added a comment -

          Thank you very much Eric Payne for helping to verify the patch. I will update the patch by adding the import.

          This patch is also now depends on YARN-3887 and YARN-4014 as mapred cli has a command to set the priority of an application at runtime. I will rebase as those tickets are committed.

          Show
          sunilg Sunil G added a comment - Thank you very much Eric Payne for helping to verify the patch. I will update the patch by adding the import. This patch is also now depends on YARN-3887 and YARN-4014 as mapred cli has a command to set the priority of an application at runtime. I will rebase as those tickets are committed.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for the patch, Sunil. Looks good in general, but I think we need to decide what to do about JobConf.getJobPriority. Currently it will return NORMAL if no priority was set but yet the job will not submit with that priority in practice. In addition JobConf.getJobPriority will crash if the priority wasn't set to one of the priority enum strings.

          Not sure there's a great fix for these. We could add a new enum, like DEFAULT or QUEUE_DEFAULT, which indicates the job priority has not been specified and we could at least do something symmetrical there. A client can see that enum returned or even set it explicitly, and the submit code can treat that enum as if no priority was set.

          The arbitrary priority value mapping to a mapred priority is trickier. We could add an enum to indicate the priority doesn't map to one of the other standard values, but that would only provide half of the picture. If a user set that priority explicitly then we wouldn't know what priority number corresponds. However we could at least map the numerical values to the standard priority enums so we're symmetrical with the toYarn behavior.

          Show
          jlowe Jason Lowe added a comment - Thanks for the patch, Sunil. Looks good in general, but I think we need to decide what to do about JobConf.getJobPriority. Currently it will return NORMAL if no priority was set but yet the job will not submit with that priority in practice. In addition JobConf.getJobPriority will crash if the priority wasn't set to one of the priority enum strings. Not sure there's a great fix for these. We could add a new enum, like DEFAULT or QUEUE_DEFAULT, which indicates the job priority has not been specified and we could at least do something symmetrical there. A client can see that enum returned or even set it explicitly, and the submit code can treat that enum as if no priority was set. The arbitrary priority value mapping to a mapred priority is trickier. We could add an enum to indicate the priority doesn't map to one of the other standard values, but that would only provide half of the picture. If a user set that priority explicitly then we wouldn't know what priority number corresponds. However we could at least map the numerical values to the standard priority enums so we're symmetrical with the toYarn behavior.
          Hide
          sunilg Sunil G added a comment -

          Thanks Jason Lowe for sharing the inputs.
          Yes, as you mentioned it wont be as perfect as we cant map some undefined numerical constants to mapred defined enums.
          Few points:

          • As you have mentioned, we could add a DEFAULT enum to indicate a case where no priority is set from client side. But now YARN has the capability to take default value, if priority is not set from client side. This can be Queue default or Cluster default (if Queue doesnt have any default or queue doesnt exist). And mapred cannot know this. So could we add a DEFAULT enum with 0 as base value for now.
          • There is another problem when we do integer mapping.
            public enum JobPriority {
              VERY_HIGH,
              HIGH,
              NORMAL,
              LOW,
              VERY_LOW;
            }
            

            We wont be able to take ordinal here because of the current order. Hence {TypeConver#toYarn}} can help in mapping this enum to a YARN model of integer priority (preferably from 1 to 5). So I can make 0 as DEFAULT (new enum)

          • I agree with your idea of having a new enum which can be returned when any other integer is been used a priority other than standard enum. Eventhough its not much helpful as its not giving exact priority value, I feel we can also have a new api public int getJobPriorityAsInteger along with existing one for more clarity in such cases.
          Show
          sunilg Sunil G added a comment - Thanks Jason Lowe for sharing the inputs. Yes, as you mentioned it wont be as perfect as we cant map some undefined numerical constants to mapred defined enums. Few points: As you have mentioned, we could add a DEFAULT enum to indicate a case where no priority is set from client side. But now YARN has the capability to take default value, if priority is not set from client side. This can be Queue default or Cluster default (if Queue doesnt have any default or queue doesnt exist). And mapred cannot know this. So could we add a DEFAULT enum with 0 as base value for now. There is another problem when we do integer mapping. public enum JobPriority { VERY_HIGH, HIGH, NORMAL, LOW, VERY_LOW; } We wont be able to take ordinal here because of the current order. Hence {TypeConver#toYarn}} can help in mapping this enum to a YARN model of integer priority (preferably from 1 to 5). So I can make 0 as DEFAULT (new enum) I agree with your idea of having a new enum which can be returned when any other integer is been used a priority other than standard enum. Eventhough its not much helpful as its not giving exact priority value, I feel we can also have a new api public int getJobPriorityAsInteger along with existing one for more clarity in such cases.
          Hide
          jlowe Jason Lowe added a comment -

          But now YARN has the capability to take default value, if priority is not set from client side.

          That's exactly what I intended for the new DEFAULT enum to indicate. If set on the client side then the job submitter shouldy avoid setting any job priority and let the cluster set an automatic value. In other words, this specific enum value should be treated as if there was no conf property set.

          So I can make 0 as DEFAULT (new enum)

          Note that we should not treat a priority of 0 as DEFAULT. As I mentioned above, DEFAULT means there is no explicit priority set, so no integer priority value should map to the DEFAULT enum.

          I agree with your idea of having a new enum which can be returned when any other integer is been used a priority other than standard enum.

          My main complaint with that approach is that it's not symmetric. That enum value cannot be set by clients, and it should be treated as an error since we cannot know which integer priority corresponds. But at least it keeps the getJobPriority call from crashing, and I don't know of a better approach offhand.

          Show
          jlowe Jason Lowe added a comment - But now YARN has the capability to take default value, if priority is not set from client side. That's exactly what I intended for the new DEFAULT enum to indicate. If set on the client side then the job submitter shouldy avoid setting any job priority and let the cluster set an automatic value. In other words, this specific enum value should be treated as if there was no conf property set. So I can make 0 as DEFAULT (new enum) Note that we should not treat a priority of 0 as DEFAULT. As I mentioned above, DEFAULT means there is no explicit priority set, so no integer priority value should map to the DEFAULT enum. I agree with your idea of having a new enum which can be returned when any other integer is been used a priority other than standard enum. My main complaint with that approach is that it's not symmetric. That enum value cannot be set by clients, and it should be treated as an error since we cannot know which integer priority corresponds. But at least it keeps the getJobPriority call from crashing, and I don't know of a better approach offhand.
          Hide
          sunilg Sunil G added a comment -

          As I mentioned above, DEFAULT means there is no explicit priority set, so no integer priority value should map to the DEFAULT enum

          Yes. That's correct, if we set some other value, it will give a wrong meaning as it can also be a valid priority in real case. I will use plain enum without any mapping as such.

          But at least it keeps the getJobPriority call from crashing

          Yes, its like handling a default case. I will try giving a more meaning name for that ENUM, so users can try new api to get exact integer mapping value for priority atleast.

          As YARN-4014 is getting in, I will handle mapred cli command also to set priority at run time in same ticket.

          Show
          sunilg Sunil G added a comment - As I mentioned above, DEFAULT means there is no explicit priority set, so no integer priority value should map to the DEFAULT enum Yes. That's correct, if we set some other value, it will give a wrong meaning as it can also be a valid priority in real case. I will use plain enum without any mapping as such. But at least it keeps the getJobPriority call from crashing Yes, its like handling a default case. I will try giving a more meaning name for that ENUM, so users can try new api to get exact integer mapping value for priority atleast. As YARN-4014 is getting in, I will handle mapred cli command also to set priority at run time in same ticket.
          Hide
          eepayne Eric Payne added a comment -

          Sunil G,

          As YARN-4014 is getting in, I will handle mapred cli command also to set priority at run time in same ticket.

          Are you saying that you will be combining pieces of this JIRA (MAPREDUCE-5870) with YARN-4014? Can you please clarify which pieces?

          Show
          eepayne Eric Payne added a comment - Sunil G , As YARN-4014 is getting in, I will handle mapred cli command also to set priority at run time in same ticket. Are you saying that you will be combining pieces of this JIRA ( MAPREDUCE-5870 ) with YARN-4014 ? Can you please clarify which pieces?
          Hide
          sunilg Sunil G added a comment -

          HI Eric Payne
          In Job class, we have an api called setJobPriority and it invokes setJobPriority api in ClientProtocol if job is running. And this api is dummy for now.
          Job.java

                ensureState(JobState.RUNNING);
                final JobPriority tmpPriority = priority;
                ugi.doAs(new PrivilegedExceptionAction<Object>() {
                  @Override
                  public Object run() throws IOException, InterruptedException {
                    cluster.getClient().setJobPriority(getJobID(), tmpPriority.toString());
                    return null;
                  }
                });
          

          ResourceMgDelegate.java

            public void setJobPriority(JobID arg0, String arg1) throws IOException,
                InterruptedException {
              return;
            }
          

          YARN-4014 introduces new api in ClientProtocol to set priority of an application at runtime. So we can use that api instead of current dummy one.

          Existing cli is mapred -set-priority <values>

          Show
          sunilg Sunil G added a comment - HI Eric Payne In Job class, we have an api called setJobPriority and it invokes setJobPriority api in ClientProtocol if job is running. And this api is dummy for now. Job.java ensureState(JobState.RUNNING); final JobPriority tmpPriority = priority; ugi.doAs( new PrivilegedExceptionAction< Object >() { @Override public Object run() throws IOException, InterruptedException { cluster.getClient().setJobPriority(getJobID(), tmpPriority.toString()); return null ; } }); ResourceMgDelegate.java public void setJobPriority(JobID arg0, String arg1) throws IOException, InterruptedException { return ; } YARN-4014 introduces new api in ClientProtocol to set priority of an application at runtime. So we can use that api instead of current dummy one. Existing cli is mapred -set-priority <values>
          Hide
          sunilg Sunil G added a comment -

          Updating a new version of patch addressing the comments. Kindly help to check the same.

          Show
          sunilg Sunil G added a comment - Updating a new version of patch addressing the comments. Kindly help to check the same.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 59s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 40s There were no new javac warning messages.
          +1 javadoc 9m 46s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 31s The applied patch generated 26 new checkstyle issues (total was 255, now 277).
          -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 21s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 0m 47s Tests passed in hadoop-mapreduce-client-common.
          -1 mapreduce tests 1m 41s Tests failed in hadoop-mapreduce-client-core.
          -1 mapreduce tests 108m 1s Tests failed in hadoop-mapreduce-client-jobclient.
              153m 27s  



          Reason Tests
          Failed unit tests hadoop.mapred.TestJobConf
            hadoop.mapreduce.TestMRJobClient



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12752221/0004-MAPREDUCE-5870.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / eee0d45
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/whitespace.txt
          hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
          hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
          hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 59s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 40s There were no new javac warning messages. +1 javadoc 9m 46s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 31s The applied patch generated 26 new checkstyle issues (total was 255, now 277). -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 21s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 0m 47s Tests passed in hadoop-mapreduce-client-common. -1 mapreduce tests 1m 41s Tests failed in hadoop-mapreduce-client-core. -1 mapreduce tests 108m 1s Tests failed in hadoop-mapreduce-client-jobclient.     153m 27s   Reason Tests Failed unit tests hadoop.mapred.TestJobConf   hadoop.mapreduce.TestMRJobClient Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752221/0004-MAPREDUCE-5870.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / eee0d45 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5952/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for updating the patch, Sunil!

          I'm not sure why we need an updateJobPriority that's separate from the existing setJobPriority call that's already there. Seems like we just need to put the code for updateApplicationPriority into ResourceMgrDelegate.setJobPriority which currently does nothing.

          Why does setJobPriority return DEFAULT if the priority appears to be a negative number but returns UNDEFINED_PRIORITY if the number is not negative? I don't recall negative priority values being special.

          We need to translate equivalent integer priorities to the corresponding enum. For example, if someone sets the priority as 5 then getJobPriority should return VERY_HIGH. Otherwise when someone sets a priority via enum they will get back UNDEFINED_PRIORITY instead of what they set which is inconsistent.

          Job.setPriority should preserve the enum name in the conf for better backwards compatibility. I think we should pass the Job.setPriority argument straight through JobConf.setJobPriority if the enum isn't UNDEFINED_PRIORITY.

          TypeConverter needs to handle the new DEFAULT priority as zero.

          Nit: Please add comments to the two new JobPriority enums explaining their usage.

          The handling of priority from JobStatus also needs to be updated. Currently the code is hardwired to return a NORMAL priority for a running job. See the way TypeConverter handles an application report and builds up a job status.

          There should be some unit tests. Minimally there should be tests to check some of the things we've discussed wrt. consistency of priority get and set. For example, setting a priority enum should return that enum on get for the old priority enums, etc. There should also be a test that verifies we are passing the priority in the app submission context appropriately and calling the appropriate API to update job priority when job.setPriority is called on a running job. Finally there should be tests to verify we are marshalling a YARN application report properly wrt. priority handling.

          Show
          jlowe Jason Lowe added a comment - Thanks for updating the patch, Sunil! I'm not sure why we need an updateJobPriority that's separate from the existing setJobPriority call that's already there. Seems like we just need to put the code for updateApplicationPriority into ResourceMgrDelegate.setJobPriority which currently does nothing. Why does setJobPriority return DEFAULT if the priority appears to be a negative number but returns UNDEFINED_PRIORITY if the number is not negative? I don't recall negative priority values being special. We need to translate equivalent integer priorities to the corresponding enum. For example, if someone sets the priority as 5 then getJobPriority should return VERY_HIGH. Otherwise when someone sets a priority via enum they will get back UNDEFINED_PRIORITY instead of what they set which is inconsistent. Job.setPriority should preserve the enum name in the conf for better backwards compatibility. I think we should pass the Job.setPriority argument straight through JobConf.setJobPriority if the enum isn't UNDEFINED_PRIORITY. TypeConverter needs to handle the new DEFAULT priority as zero. Nit: Please add comments to the two new JobPriority enums explaining their usage. The handling of priority from JobStatus also needs to be updated. Currently the code is hardwired to return a NORMAL priority for a running job. See the way TypeConverter handles an application report and builds up a job status. There should be some unit tests. Minimally there should be tests to check some of the things we've discussed wrt. consistency of priority get and set. For example, setting a priority enum should return that enum on get for the old priority enums, etc. There should also be a test that verifies we are passing the priority in the app submission context appropriately and calling the appropriate API to update job priority when job.setPriority is called on a running job. Finally there should be tests to verify we are marshalling a YARN application report properly wrt. priority handling.
          Hide
          sunilg Sunil G added a comment -

          Thank you Jason Lowe for the detailed comments. I will work on these items and will update a patch shortly.

          Show
          sunilg Sunil G added a comment - Thank you Jason Lowe for the detailed comments. I will work on these items and will update a patch shortly.
          Hide
          sunilg Sunil G added a comment -

          Thank you Jason Lowe for the comments. i have uploaded patch addressing the same. Kindly help to check the same.

          Show
          sunilg Sunil G added a comment - Thank you Jason Lowe for the comments. i have uploaded patch addressing the same. Kindly help to check the same.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 0s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 6 new or modified test files.
          +1 javac 7m 42s There were no new javac warning messages.
          +1 javadoc 9m 49s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 32s The applied patch generated 49 new checkstyle issues (total was 367, now 412).
          -1 whitespace 0m 8s The patch has 9 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 0m 46s Tests passed in hadoop-mapreduce-client-common.
          +1 mapreduce tests 1m 45s Tests passed in hadoop-mapreduce-client-core.
          -1 mapreduce tests 70m 53s Tests failed in hadoop-mapreduce-client-jobclient.
              116m 35s  



          Reason Tests
          Failed unit tests hadoop.mapred.TestReduceFetch
            hadoop.mapred.TestNetworkedJob
            hadoop.mapred.TestJobName
            hadoop.mapred.TestLazyOutput
            hadoop.mapreduce.TestMRJobClient
          Timed out tests org.apache.hadoop.mapred.TestMiniMRClasspath
            org.apache.hadoop.mapred.TestLocalMRNotification
            org.apache.hadoop.mapred.TestLineRecordReaderJobs



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12753821/0005-MAPREDUCE-5870.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 7d6687f
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/whitespace.txt
          hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
          hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
          hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 0s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 6 new or modified test files. +1 javac 7m 42s There were no new javac warning messages. +1 javadoc 9m 49s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 32s The applied patch generated 49 new checkstyle issues (total was 367, now 412). -1 whitespace 0m 8s The patch has 9 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 0m 46s Tests passed in hadoop-mapreduce-client-common. +1 mapreduce tests 1m 45s Tests passed in hadoop-mapreduce-client-core. -1 mapreduce tests 70m 53s Tests failed in hadoop-mapreduce-client-jobclient.     116m 35s   Reason Tests Failed unit tests hadoop.mapred.TestReduceFetch   hadoop.mapred.TestNetworkedJob   hadoop.mapred.TestJobName   hadoop.mapred.TestLazyOutput   hadoop.mapreduce.TestMRJobClient Timed out tests org.apache.hadoop.mapred.TestMiniMRClasspath   org.apache.hadoop.mapred.TestLocalMRNotification   org.apache.hadoop.mapred.TestLineRecordReaderJobs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753821/0005-MAPREDUCE-5870.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7d6687f checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5968/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Attaching a cleaner patch after fixing checkstyle, whitespace etc. test case failure is due to non-compilation of TypeConverter.java. I will observe the test cases in this patch too.

          Show
          sunilg Sunil G added a comment - Attaching a cleaner patch after fixing checkstyle, whitespace etc. test case failure is due to non-compilation of TypeConverter.java. I will observe the test cases in this patch too.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 30s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 6 new or modified test files.
          +1 javac 7m 56s There were no new javac warning messages.
          -1 javadoc 10m 8s The applied patch generated 2 additional warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 34s The applied patch generated 8 new checkstyle issues (total was 367, now 371).
          -1 whitespace 0m 9s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 3m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 0m 47s Tests passed in hadoop-mapreduce-client-common.
          +1 mapreduce tests 1m 49s Tests passed in hadoop-mapreduce-client-core.
          -1 mapreduce tests 96m 38s Tests failed in hadoop-mapreduce-client-jobclient.
              143m 43s  



          Reason Tests
          Failed unit tests hadoop.mapreduce.TestValueIterReset
            hadoop.mapred.TestYARNRunner
            hadoop.mapreduce.TestChild
            hadoop.mapreduce.TestMapReduceLazyOutput
            hadoop.mapreduce.TestMRJobClient
            hadoop.mapred.TestNetworkedJob



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12753984/0006-MAPREDUCE-5870.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 9a87f81
          javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/diffJavadocWarnings.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/whitespace.txt
          hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt
          hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
          hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 30s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 6 new or modified test files. +1 javac 7m 56s There were no new javac warning messages. -1 javadoc 10m 8s The applied patch generated 2 additional warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 34s The applied patch generated 8 new checkstyle issues (total was 367, now 371). -1 whitespace 0m 9s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 0m 47s Tests passed in hadoop-mapreduce-client-common. +1 mapreduce tests 1m 49s Tests passed in hadoop-mapreduce-client-core. -1 mapreduce tests 96m 38s Tests failed in hadoop-mapreduce-client-jobclient.     143m 43s   Reason Tests Failed unit tests hadoop.mapreduce.TestValueIterReset   hadoop.mapred.TestYARNRunner   hadoop.mapreduce.TestChild   hadoop.mapreduce.TestMapReduceLazyOutput   hadoop.mapreduce.TestMRJobClient   hadoop.mapred.TestNetworkedJob Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12753984/0006-MAPREDUCE-5870.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 9a87f81 javadoc https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/diffJavadocWarnings.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-common test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/testrun_hadoop-mapreduce-client-common.txt hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5969/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for updating the patch!

          We cannot change ClientProtocol.setJobPriority to take an integer rather than a String. That's a backwards-incompatible change.

          Why are we adding ClientProtocol.getJobPriority? We are already returning the job priority via getJobStatus, so I'm a little confused on why we are extending the protocol to get information the client can already retrieve.

          Does it make more sense to have LocalJobRunner.getJobPriority return a normal or default priority? Returning null seems disruptive, as priorities really don't matter in the local job mode scenario. In a similar sense, I'm wondering if we should have LocalJobRunner.setJobPriority simply track or ignore the priority rather than explode. Since it already exploded for setJobPriority before we can tackle that in a separate JIRA if desired.

          Why does JobConf.setPriorityAsInteger take a boxed type rather than a normal int and call Integer.toString?

          I think the new TypeConverter functions are a little too generically named. It's a bit dangerous to have a toYarn that takes something as generic as a String and returns an int – it could easily me misused in other contexts that have nothing to do with job priorities. I'd name these fromYarnPriority, toYarnPriority, etc. so there's no chance of accidental collision.

          convertPriorityToInteger needs to handle the case where the string is itself an integer, otherwise setJobPriorityAsInteger followed by getJobPriorityAsInteger doesn't work as expected. Trying to do JobPriority.valueOf on a string that may not contain a valid enum is going to throw an IllegalArgumentException. Please add unit tests for the various get/set as integer scenarios on both Job and JobConf to make sure we're handling them properly.

          Why was the call to testChangingJobPriority removed from TestMRJobClient? The function is still there, although in practice is never called.

          Show
          jlowe Jason Lowe added a comment - Thanks for updating the patch! We cannot change ClientProtocol.setJobPriority to take an integer rather than a String. That's a backwards-incompatible change. Why are we adding ClientProtocol.getJobPriority? We are already returning the job priority via getJobStatus, so I'm a little confused on why we are extending the protocol to get information the client can already retrieve. Does it make more sense to have LocalJobRunner.getJobPriority return a normal or default priority? Returning null seems disruptive, as priorities really don't matter in the local job mode scenario. In a similar sense, I'm wondering if we should have LocalJobRunner.setJobPriority simply track or ignore the priority rather than explode. Since it already exploded for setJobPriority before we can tackle that in a separate JIRA if desired. Why does JobConf.setPriorityAsInteger take a boxed type rather than a normal int and call Integer.toString? I think the new TypeConverter functions are a little too generically named. It's a bit dangerous to have a toYarn that takes something as generic as a String and returns an int – it could easily me misused in other contexts that have nothing to do with job priorities. I'd name these fromYarnPriority, toYarnPriority, etc. so there's no chance of accidental collision. convertPriorityToInteger needs to handle the case where the string is itself an integer, otherwise setJobPriorityAsInteger followed by getJobPriorityAsInteger doesn't work as expected. Trying to do JobPriority.valueOf on a string that may not contain a valid enum is going to throw an IllegalArgumentException. Please add unit tests for the various get/set as integer scenarios on both Job and JobConf to make sure we're handling them properly. Why was the call to testChangingJobPriority removed from TestMRJobClient? The function is still there, although in practice is never called.
          Hide
          sunilg Sunil G added a comment -

          Thank you Jason Lowe for sharing the comments.

          As discussed offline, I had doubts on couple of points here.

          We are already returning the job priority via getJobStatus

          As I see, we are invoking YARNRunner#getJobStatus to get the job status while we try to call Job#getPriority.
          And in ClientServiceDelegate#getJobStatus, JobStatus is created from JobReport (from AM).

            public JobStatus getJobStatus(JobID oldJobID) throws IOException {
              org.apache.hadoop.mapreduce.v2.api.records.JobId jobId =
                TypeConverter.toYarn(oldJobID);
              GetJobReportRequest request =
                  recordFactory.newRecordInstance(GetJobReportRequest.class);
              request.setJobId(jobId);
              JobReport report = ((GetJobReportResponse) invoke("getJobReport",
                  GetJobReportRequest.class, request)).getJobReport();
          ...
          ...
              jobStatus = TypeConverter.fromYarn(report, url);
          ...
          

          As this JobReport doesnt have priority information, current code in TypeConverter#fromYarn is to hardcode the status.

            public static JobStatus fromYarn(JobReport jobreport, String trackingUrl) {
              JobPriority jobPriority = JobPriority.NORMAL;
              JobStatus jobStatus = new org.apache.hadoop.mapred.JobStatus(
                  fromYarn(jobreport.getJobId()), jobreport.getSetupProgress(), jobreport
          ....
          

          AppReport has the information about the priority of the application. So a second call has to be made to fetch the same from YarnRunner#getJobStatus. And it can be set to JobStatus separately.
          Kindly help to share your thoughts here if I am missing something important.

          Why was the call to testChangingJobPriority removed from TestMRJobClient?

          TestMRJobClient#testChangingJobPriority was invoked after job was completed. And YARN throws excpetion for change priority application if its not RUNNING or ACCEPTED. As updated offline, I agree that YARN need not have to throw excpetion if the app is completed, rather it can be skipped. I will file a ticket in YARN to fix the same. Thoughts?

          For LocalJobRunner exception case, I will file another ticket to fix the exception and to support correct priority.
          Also I will add all possible test cases for Job and JobConf to see all possible integer and enums are set/get correctly.
          Thank you for the support.

          Show
          sunilg Sunil G added a comment - Thank you Jason Lowe for sharing the comments. As discussed offline, I had doubts on couple of points here. We are already returning the job priority via getJobStatus As I see, we are invoking YARNRunner#getJobStatus to get the job status while we try to call Job#getPriority . And in ClientServiceDelegate#getJobStatus , JobStatus is created from JobReport (from AM). public JobStatus getJobStatus(JobID oldJobID) throws IOException { org.apache.hadoop.mapreduce.v2.api.records.JobId jobId = TypeConverter.toYarn(oldJobID); GetJobReportRequest request = recordFactory.newRecordInstance(GetJobReportRequest.class); request.setJobId(jobId); JobReport report = ((GetJobReportResponse) invoke( "getJobReport" , GetJobReportRequest.class, request)).getJobReport(); ... ... jobStatus = TypeConverter.fromYarn(report, url); ... As this JobReport doesnt have priority information, current code in TypeConverter#fromYarn is to hardcode the status. public static JobStatus fromYarn(JobReport jobreport, String trackingUrl) { JobPriority jobPriority = JobPriority.NORMAL; JobStatus jobStatus = new org.apache.hadoop.mapred.JobStatus( fromYarn(jobreport.getJobId()), jobreport.getSetupProgress(), jobreport .... AppReport has the information about the priority of the application. So a second call has to be made to fetch the same from YarnRunner#getJobStatus . And it can be set to JobStatus separately. Kindly help to share your thoughts here if I am missing something important. Why was the call to testChangingJobPriority removed from TestMRJobClient? TestMRJobClient#testChangingJobPriority was invoked after job was completed. And YARN throws excpetion for change priority application if its not RUNNING or ACCEPTED. As updated offline, I agree that YARN need not have to throw excpetion if the app is completed, rather it can be skipped. I will file a ticket in YARN to fix the same. Thoughts? For LocalJobRunner exception case, I will file another ticket to fix the exception and to support correct priority. Also I will add all possible test cases for Job and JobConf to see all possible integer and enums are set/get correctly. Thank you for the support.
          Hide
          jlowe Jason Lowe added a comment -

          AppReport has the information about the priority of the application. So a second call has to be made to fetch the same from YarnRunner#getJobStatus. And it can be set to JobStatus separately.

          The problem is that the ApplicationMaster doesn't actually know what its job priority actually is, as that's a concept tracked by the RM not the AM. So to report the priority we'd have to make a separate call to the RM to determine the priority and meld that into the rest of the information returned for the job status. I'm not thrilled with the thought of adding an additional call to the RM for every time a client wants to get the job status, especially for a field that most clients won't need. Clients often bang on the job status to see when a job is done, and that means they'll start banging on the RM as well if we make the delegate start hitting both the AM and the RM to generate the JobStatus.

          There's only two ways I can currently think of to get around that. Either we need to obsolete the priority field from the job status and provide a separate call to get the priority when the client really wants to know it, or we have to find a way for the AM to know its job priority so it can return it in its JobStatus responses. For the latter we could have the RM send it down in heartbeat responses, but there would be a delay between when a client updates the priority and the AM reports the updated value. We could have the AM proactively ask the RM what its priority is, but then we're back to banging on the RM for every job status call (only this time from AMs instead of the client, but same number of RM calls overall).

          Any other thoughts on what we should do here? I'm not thrilled about the thought of ditching the priority in the JobStatus field. However it's been worthless ever since we went to YARN (i.e.: broken since 0.23 and 2.x), so it's tempting to declare it officially obsolete (as it in practice already is) and provide a different API to retrieve it.

          And YARN throws excpetion for change priority application if its not RUNNING or ACCEPTED.

          Yes, this to me is a bug. We don't throw exceptions when clients try to kill apps that have terminated, and similarly we shouldn't throw exceptions if clients try to change the priority of terminated apps.

          Show
          jlowe Jason Lowe added a comment - AppReport has the information about the priority of the application. So a second call has to be made to fetch the same from YarnRunner#getJobStatus. And it can be set to JobStatus separately. The problem is that the ApplicationMaster doesn't actually know what its job priority actually is, as that's a concept tracked by the RM not the AM. So to report the priority we'd have to make a separate call to the RM to determine the priority and meld that into the rest of the information returned for the job status. I'm not thrilled with the thought of adding an additional call to the RM for every time a client wants to get the job status, especially for a field that most clients won't need. Clients often bang on the job status to see when a job is done, and that means they'll start banging on the RM as well if we make the delegate start hitting both the AM and the RM to generate the JobStatus. There's only two ways I can currently think of to get around that. Either we need to obsolete the priority field from the job status and provide a separate call to get the priority when the client really wants to know it, or we have to find a way for the AM to know its job priority so it can return it in its JobStatus responses. For the latter we could have the RM send it down in heartbeat responses, but there would be a delay between when a client updates the priority and the AM reports the updated value. We could have the AM proactively ask the RM what its priority is, but then we're back to banging on the RM for every job status call (only this time from AMs instead of the client, but same number of RM calls overall). Any other thoughts on what we should do here? I'm not thrilled about the thought of ditching the priority in the JobStatus field. However it's been worthless ever since we went to YARN (i.e.: broken since 0.23 and 2.x), so it's tempting to declare it officially obsolete (as it in practice already is) and provide a different API to retrieve it. And YARN throws excpetion for change priority application if its not RUNNING or ACCEPTED. Yes, this to me is a bug. We don't throw exceptions when clients try to kill apps that have terminated, and similarly we shouldn't throw exceptions if clients try to change the priority of terminated apps.
          Hide
          eepayne Eric Payne added a comment -

          There's only two ways I can currently think of to get around that. Either we need to obsolete the priority field from the job status and provide a separate call to get the priority when the client really wants to know it, or we have to find a way for the AM to know its job priority so it can return it in its JobStatus responses. For the latter we could have the RM send it down in heartbeat responses, but there would be a delay between when a client updates the priority and the AM reports the updated value.

          Jason Lowe and Sunil G,
          I would vote for the second option. That is, update the priority in the RM and then tell the AM about it when it hearbeats in. I think it would be fine to have the short delay between when the priority was updated and the AM knows about it. I can't foresee a use case where the client would atomically need to know about the change. The one thing that might cause confusion is if the client set the priority and immediately read the priority which might still be the old value. This may cause confusion, but as long as this behavior is well documented it should be fine.

          Show
          eepayne Eric Payne added a comment - There's only two ways I can currently think of to get around that. Either we need to obsolete the priority field from the job status and provide a separate call to get the priority when the client really wants to know it, or we have to find a way for the AM to know its job priority so it can return it in its JobStatus responses. For the latter we could have the RM send it down in heartbeat responses, but there would be a delay between when a client updates the priority and the AM reports the updated value. Jason Lowe and Sunil G , I would vote for the second option. That is, update the priority in the RM and then tell the AM about it when it hearbeats in. I think it would be fine to have the short delay between when the priority was updated and the AM knows about it. I can't foresee a use case where the client would atomically need to know about the change. The one thing that might cause confusion is if the client set the priority and immediately read the priority which might still be the old value. This may cause confusion, but as long as this behavior is well documented it should be fine.
          Hide
          sunilg Sunil G added a comment -

          Hi Jason Lowe and Eric Payne
          Thank you very much for the comments.

          Yes. Option 2 will be helpful to cover the existing functionality. And Priority is logically a part of JobStatus, so its good to keep that maximum possible way. However, if AM can get to know the Application Priority, is there any other advantage AM will have other than updating JobStatus. Still its good to do this way as we can avoid 2 RPC calls during every status query.

          Show
          sunilg Sunil G added a comment - Hi Jason Lowe and Eric Payne Thank you very much for the comments. Yes. Option 2 will be helpful to cover the existing functionality. And Priority is logically a part of JobStatus, so its good to keep that maximum possible way. However, if AM can get to know the Application Priority, is there any other advantage AM will have other than updating JobStatus. Still its good to do this way as we can avoid 2 RPC calls during every status query.
          Hide
          sunilg Sunil G added a comment -

          Hi Jason Lowe Eric Payne Jian He

          As discussed if option-2 is fine, I will raise a separate ticket in YARN to handle RM-AM update of priority (through heartbeat).
          And I will separate the JobStatus priority update from this ticket for now. And once YARN is done, we can handle JobStatus priority update in correct way.

          Could you share your opinion on same.

          Show
          sunilg Sunil G added a comment - Hi Jason Lowe Eric Payne Jian He As discussed if option-2 is fine, I will raise a separate ticket in YARN to handle RM-AM update of priority (through heartbeat). And I will separate the JobStatus priority update from this ticket for now. And once YARN is done, we can handle JobStatus priority update in correct way. Could you share your opinion on same.
          Hide
          eepayne Eric Payne added a comment -

          As discussed if option-2 is fine, I will raise a separate ticket in YARN to handle RM-AM update of priority (through heartbeat).
          And I will separate the JobStatus priority update from this ticket for now.

          Thanks, Sunil G. If I understand correctly, you will update the patch for this JIRA so that the AM will not query the RM for it's job priority, and then in another JIRA, mkae the change to have the RM tell the AM its priority as part a heartbeat ack. Is that correct? I just want to make sure that this JIRA doesn't add that extra load on the RM for an AM JobStatus query.

          Show
          eepayne Eric Payne added a comment - As discussed if option-2 is fine, I will raise a separate ticket in YARN to handle RM-AM update of priority (through heartbeat). And I will separate the JobStatus priority update from this ticket for now. Thanks, Sunil G . If I understand correctly, you will update the patch for this JIRA so that the AM will not query the RM for it's job priority, and then in another JIRA, mkae the change to have the RM tell the AM its priority as part a heartbeat ack. Is that correct? I just want to make sure that this JIRA doesn't add that extra load on the RM for an AM JobStatus query.
          Hide
          sunilg Sunil G added a comment -

          Yes. I thought of not updating JobStatus for priority from RM here for now. Once we do in RM, I will update priority to JobStatus from updated JobReport. Looping Jian He

          Show
          sunilg Sunil G added a comment - Yes. I thought of not updating JobStatus for priority from RM here for now. Once we do in RM, I will update priority to JobStatus from updated JobReport. Looping Jian He
          Hide
          jianhe Jian He added a comment -

          I earlier thought we can keep backward compatible with the enum priority, but now am thinking the value of doing this. This does bring extra complexity to support both. Jason Lowe, do you know if there are many apps from MR1 are actually expecting this enum based priority to work ? Since priority is never supported since hadoop 2 for such a long time, I'm thinking if we can deprecate the old API and claim only support integers to be simple and clear.

          Show
          jianhe Jian He added a comment - I earlier thought we can keep backward compatible with the enum priority, but now am thinking the value of doing this. This does bring extra complexity to support both. Jason Lowe , do you know if there are many apps from MR1 are actually expecting this enum based priority to work ? Since priority is never supported since hadoop 2 for such a long time, I'm thinking if we can deprecate the old API and claim only support integers to be simple and clear.
          Hide
          jlowe Jason Lowe added a comment -

          I honestly don't know how much the priority APIs were used from MRv1. However given that there were priority APIs in MRv1 that's the most logical thing people migrating from Hadoop 1 will expect to use when they hear priorities are finally working in Hadoop 2.

          If we decide to deprecate the old MRv1 priority API, should we avoid using mapreduce.job.priority (was mapred.job.priority)? Wondering if jobs that were working in Hadoop 2 suddenly start failing during job submit because someone was specifying something for that property that was always ignored and now the new code barfs on any non-integer found there. However I'm not excited about the confusion of having two similar properties to set the job priority, one that works and one that does not.

          Show
          jlowe Jason Lowe added a comment - I honestly don't know how much the priority APIs were used from MRv1. However given that there were priority APIs in MRv1 that's the most logical thing people migrating from Hadoop 1 will expect to use when they hear priorities are finally working in Hadoop 2. If we decide to deprecate the old MRv1 priority API, should we avoid using mapreduce.job.priority (was mapred.job.priority)? Wondering if jobs that were working in Hadoop 2 suddenly start failing during job submit because someone was specifying something for that property that was always ignored and now the new code barfs on any non-integer found there. However I'm not excited about the confusion of having two similar properties to set the job priority, one that works and one that does not.
          Hide
          jianhe Jian He added a comment -

          I see, I'm ok to keep it supporting the enum as that's what I originally thought. Just want to bring this up.

          Show
          jianhe Jian He added a comment - I see, I'm ok to keep it supporting the enum as that's what I originally thought. Just want to bring this up.
          Hide
          sunilg Sunil G added a comment -

          Hi Jason Lowe Jian He Eric Payne

          I thought of bringing up a point we discussed earlier as I have one more concern. To keep priority in JobStatus, and also to contact only AM (not RM) to get JobStatus, we had discussed on updating priority to AM from RM through heartbeat.
          However, other than updating JobStatus, I am not finding any other good useful reason for AM to know its own priority in RM. Do you see any more advantages for AM knowing its priority(may be other than MapReduce AM also).

          Show
          sunilg Sunil G added a comment - Hi Jason Lowe Jian He Eric Payne I thought of bringing up a point we discussed earlier as I have one more concern. To keep priority in JobStatus, and also to contact only AM (not RM) to get JobStatus, we had discussed on updating priority to AM from RM through heartbeat. However, other than updating JobStatus, I am not finding any other good useful reason for AM to know its own priority in RM. Do you see any more advantages for AM knowing its priority(may be other than MapReduce AM also).
          Hide
          jlowe Jason Lowe added a comment -

          I too struggle with whether it's worth going through all this trouble. I honestly don't know how much the MRv1 priority API was used and therefore the impact of completely abandoning it.

          If we are going to keep supporting the MRv1 priority API, what are your thoughts about handling the reporting of app priority via JobStatus from the AM? I really don't want YARNRunner hitting the RM for every status update, as that will put a lot more RPC load on the RM than we have today. But the alternative means we either need to have the AM report a delayed view of the priority (e.g.: send priority in heartbeat response, AM may be out-of-sync by up to one heartbeat interval), or we have to deprecate that part of the MRv1 API.

          I think I'm OK with having the priority reported in the AM heartbeat. It will be somewhat delayed with reality when it changes, but it should be relatively rare for it to be out of sync more than a few seconds. The priority would have to have changed while the AM is having difficulty contacting the RM. If that persists too long either the RM will expire the AM and kill it or the AM will give up on the RM and exit. Having the AM know the priority could also help with app frameworks that want to report this in their AM-provided UI.

          Show
          jlowe Jason Lowe added a comment - I too struggle with whether it's worth going through all this trouble. I honestly don't know how much the MRv1 priority API was used and therefore the impact of completely abandoning it. If we are going to keep supporting the MRv1 priority API, what are your thoughts about handling the reporting of app priority via JobStatus from the AM? I really don't want YARNRunner hitting the RM for every status update, as that will put a lot more RPC load on the RM than we have today. But the alternative means we either need to have the AM report a delayed view of the priority (e.g.: send priority in heartbeat response, AM may be out-of-sync by up to one heartbeat interval), or we have to deprecate that part of the MRv1 API. I think I'm OK with having the priority reported in the AM heartbeat. It will be somewhat delayed with reality when it changes, but it should be relatively rare for it to be out of sync more than a few seconds. The priority would have to have changed while the AM is having difficulty contacting the RM. If that persists too long either the RM will expire the AM and kill it or the AM will give up on the RM and exit. Having the AM know the priority could also help with app frameworks that want to report this in their AM-provided UI.
          Hide
          sunilg Sunil G added a comment -

          Thank you Jason Lowe for the detailed information.
          I will raise a YARN ticket which will update the AllocatedResponse to carry application priority. Also with another MR ticket, i will update JobStatus with priority which came from AllocatedResponse.

          Show
          sunilg Sunil G added a comment - Thank you Jason Lowe for the detailed information. I will raise a YARN ticket which will update the AllocatedResponse to carry application priority. Also with another MR ticket, i will update JobStatus with priority which came from AllocatedResponse.
          Hide
          sunilg Sunil G added a comment -

          As MAPREDUCE-6515 is committed, we can get the priority from AM in JobStatus. Updating the patch with this support.

          Show
          sunilg Sunil G added a comment - As MAPREDUCE-6515 is committed, we can get the priority from AM in JobStatus. Updating the patch with this support.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 6s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 1s The patch appears to include 5 new or modified test files.
          +1 mvninstall 3m 11s trunk passed
          +1 compile 2m 24s trunk passed with JDK v1.8.0_60
          +1 compile 2m 24s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 22s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 2m 30s trunk passed
          +1 javadoc 0m 56s trunk passed with JDK v1.8.0_60
          +1 javadoc 1m 7s trunk passed with JDK v1.7.0_79
          +1 mvninstall 1m 12s the patch passed
          +1 compile 2m 24s the patch passed with JDK v1.8.0_60
          +1 javac 2m 24s the patch passed
          +1 compile 2m 24s the patch passed with JDK v1.7.0_79
          +1 javac 2m 24s the patch passed
          -1 checkstyle 0m 20s Patch generated 22 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 380, now 397).
          +1 mvneclipse 0m 41s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 2s the patch passed
          +1 javadoc 0m 58s the patch passed with JDK v1.8.0_60
          +1 javadoc 1m 6s the patch passed with JDK v1.7.0_79
          +1 unit 0m 40s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_60.
          -1 unit 1m 40s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          -1 unit 103m 22s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_60.
          +1 unit 0m 51s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79.
          -1 unit 2m 0s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 unit 103m 18s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 34s Patch generated 1 ASF License warnings.
          239m 46s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
            hadoop.mapred.TestMRTimelineEventHandling
            hadoop.mapreduce.v2.TestMRJobs
            hadoop.mapred.TestResourceMgrDelegate
            hadoop.mapreduce.v2.TestMRJobsWithProfiler
            hadoop.mapreduce.v2.TestUberAM
            hadoop.mapreduce.TestMRJobClient
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
            hadoop.mapred.TestMRTimelineEventHandling
            hadoop.mapreduce.v2.TestMRJobs
            hadoop.mapred.TestResourceMgrDelegate
            hadoop.mapreduce.v2.TestUberAM
            hadoop.mapreduce.TestMRJobClient



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-30
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769716/0007-MAPREDUCE-5870.patch
          JIRA Issue MAPREDUCE-5870
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux de43d808064a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh
          git revision trunk / ce31b22
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 6s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 1s The patch appears to include 5 new or modified test files. +1 mvninstall 3m 11s trunk passed +1 compile 2m 24s trunk passed with JDK v1.8.0_60 +1 compile 2m 24s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 22s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 2m 30s trunk passed +1 javadoc 0m 56s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 7s trunk passed with JDK v1.7.0_79 +1 mvninstall 1m 12s the patch passed +1 compile 2m 24s the patch passed with JDK v1.8.0_60 +1 javac 2m 24s the patch passed +1 compile 2m 24s the patch passed with JDK v1.7.0_79 +1 javac 2m 24s the patch passed -1 checkstyle 0m 20s Patch generated 22 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 380, now 397). +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 2s the patch passed +1 javadoc 0m 58s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 6s the patch passed with JDK v1.7.0_79 +1 unit 0m 40s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_60. -1 unit 1m 40s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. -1 unit 103m 22s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_60. +1 unit 0m 51s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79. -1 unit 2m 0s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 unit 103m 18s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 34s Patch generated 1 ASF License warnings. 239m 46s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager   hadoop.mapred.TestMRTimelineEventHandling   hadoop.mapreduce.v2.TestMRJobs   hadoop.mapred.TestResourceMgrDelegate   hadoop.mapreduce.v2.TestMRJobsWithProfiler   hadoop.mapreduce.v2.TestUberAM   hadoop.mapreduce.TestMRJobClient JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager   hadoop.mapred.TestMRTimelineEventHandling   hadoop.mapreduce.v2.TestMRJobs   hadoop.mapred.TestResourceMgrDelegate   hadoop.mapreduce.v2.TestUberAM   hadoop.mapreduce.TestMRJobClient Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-30 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769716/0007-MAPREDUCE-5870.patch JIRA Issue MAPREDUCE-5870 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux de43d808064a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh git revision trunk / ce31b22 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6102/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Test case failures looks related. I will fix them in the next patch.

          Show
          sunilg Sunil G added a comment - Test case failures looks related. I will fix them in the next patch.
          Hide
          sunilg Sunil G added a comment -

          Attaching an updated patch correcting test failures.

          Show
          sunilg Sunil G added a comment - Attaching an updated patch correcting test failures.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          +1 mvninstall 3m 6s trunk passed
          +1 compile 2m 24s trunk passed with JDK v1.8.0_60
          +1 compile 2m 22s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 21s trunk passed
          +1 mvneclipse 1m 4s trunk passed
          +1 findbugs 3m 14s trunk passed
          +1 javadoc 1m 17s trunk passed with JDK v1.8.0_60
          +1 javadoc 1m 23s trunk passed with JDK v1.7.0_79
          +1 mvninstall 1m 37s the patch passed
          +1 compile 2m 14s the patch passed with JDK v1.8.0_60
          +1 javac 2m 14s the patch passed
          +1 compile 2m 15s the patch passed with JDK v1.7.0_79
          +1 javac 2m 15s the patch passed
          -1 checkstyle 0m 18s Patch generated 10 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 380, now 385).
          +1 mvneclipse 0m 53s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 findbugs 3m 40s the patch passed
          +1 javadoc 1m 8s the patch passed with JDK v1.8.0_60
          +1 javadoc 1m 20s the patch passed with JDK v1.7.0_79
          +1 unit 8m 59s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_60.
          +1 unit 0m 37s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_60.
          -1 unit 1m 33s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          -1 unit 105m 49s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_60.
          +1 unit 10m 8s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79.
          +1 unit 0m 51s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79.
          -1 unit 2m 0s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 unit 107m 24s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 31s Patch generated 1 ASF License warnings.
          268m 19s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
            hadoop.mapreduce.v2.TestMRJobsWithProfiler
            hadoop.mapreduce.v2.TestUberAM
            hadoop.mapred.TestMRTimelineEventHandling
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
            hadoop.mapreduce.v2.TestUberAM
            hadoop.mapred.TestMRTimelineEventHandling



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-01
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769988/0008-MAPREDUCE-5870.patch
          JIRA Issue MAPREDUCE-5870
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 538ae0db9946 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh
          git revision trunk / 7fd6416
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 224MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 3m 6s trunk passed +1 compile 2m 24s trunk passed with JDK v1.8.0_60 +1 compile 2m 22s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 21s trunk passed +1 mvneclipse 1m 4s trunk passed +1 findbugs 3m 14s trunk passed +1 javadoc 1m 17s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 23s trunk passed with JDK v1.7.0_79 +1 mvninstall 1m 37s the patch passed +1 compile 2m 14s the patch passed with JDK v1.8.0_60 +1 javac 2m 14s the patch passed +1 compile 2m 15s the patch passed with JDK v1.7.0_79 +1 javac 2m 15s the patch passed -1 checkstyle 0m 18s Patch generated 10 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 380, now 385). +1 mvneclipse 0m 53s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 3m 40s the patch passed +1 javadoc 1m 8s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 20s the patch passed with JDK v1.7.0_79 +1 unit 8m 59s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_60. +1 unit 0m 37s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_60. -1 unit 1m 33s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. -1 unit 105m 49s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_60. +1 unit 10m 8s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79. +1 unit 0m 51s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79. -1 unit 2m 0s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 unit 107m 24s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 31s Patch generated 1 ASF License warnings. 268m 19s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager   hadoop.mapreduce.v2.TestMRJobsWithProfiler   hadoop.mapreduce.v2.TestUberAM   hadoop.mapred.TestMRTimelineEventHandling JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager   hadoop.mapreduce.v2.TestUberAM   hadoop.mapred.TestMRTimelineEventHandling Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-01 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769988/0008-MAPREDUCE-5870.patch JIRA Issue MAPREDUCE-5870 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 538ae0db9946 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh git revision trunk / 7fd6416 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 224MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6106/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          TestUberAM failure appears to be related, and I'm not sure about the others.

          Show
          jlowe Jason Lowe added a comment - TestUberAM failure appears to be related, and I'm not sure about the others.
          Hide
          sunilg Sunil G added a comment -

          Thank You Jason Lowe
          Yes. TestUberAM failures were related. Other test cases are failing even w/o the patch in my local. I will see one more jenkins result with this patch, and if fails, will raise a different test failure ticket to track the same.

          Show
          sunilg Sunil G added a comment - Thank You Jason Lowe Yes. TestUberAM failures were related. Other test cases are failing even w/o the patch in my local. I will see one more jenkins result with this patch, and if fails, will raise a different test failure ticket to track the same.
          Hide
          sunilg Sunil G added a comment -

          Kicking Jenkins.

          Show
          sunilg Sunil G added a comment - Kicking Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          +1 mvninstall 4m 1s trunk passed
          +1 compile 2m 51s trunk passed with JDK v1.8.0_66
          +1 compile 2m 51s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 25s trunk passed
          +1 mvneclipse 1m 7s trunk passed
          +1 findbugs 3m 32s trunk passed
          +1 javadoc 1m 31s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 36s trunk passed with JDK v1.7.0_79
          +1 mvninstall 1m 45s the patch passed
          +1 compile 2m 57s the patch passed with JDK v1.8.0_66
          +1 javac 2m 57s the patch passed
          +1 compile 2m 52s the patch passed with JDK v1.7.0_79
          +1 javac 2m 52s the patch passed
          -1 checkstyle 0m 25s Patch generated 9 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 381, now 384).
          +1 mvneclipse 1m 9s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 4m 45s the patch passed
          +1 javadoc 1m 43s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 47s the patch passed with JDK v1.7.0_79
          -1 unit 10m 7s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66.
          +1 unit 0m 47s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66.
          -1 unit 2m 4s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66.
          -1 unit 121m 0s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_66.
          +1 unit 10m 19s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79.
          +1 unit 0m 58s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79.
          -1 unit 2m 12s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 unit 0m 19s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 24s Patch generated 1 ASF License warnings.
          185m 33s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
            hadoop.mapreduce.v2.TestUberAM
            hadoop.mapreduce.v2.TestMRJobsWithProfiler
            hadoop.mapred.TestMRTimelineEventHandling
          JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.mapreduce.v2.app.TestMRAppComponentDependencies
            org.apache.hadoop.mapreduce.TestLargeSort
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-05
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770757/0009-MAPREDUCE-5870.patch
          JIRA Issue MAPREDUCE-5870
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 47da65576f39 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-e8bd3ad/precommit/personality/hadoop.sh
          git revision trunk / 5667129
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 4m 1s trunk passed +1 compile 2m 51s trunk passed with JDK v1.8.0_66 +1 compile 2m 51s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 25s trunk passed +1 mvneclipse 1m 7s trunk passed +1 findbugs 3m 32s trunk passed +1 javadoc 1m 31s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 36s trunk passed with JDK v1.7.0_79 +1 mvninstall 1m 45s the patch passed +1 compile 2m 57s the patch passed with JDK v1.8.0_66 +1 javac 2m 57s the patch passed +1 compile 2m 52s the patch passed with JDK v1.7.0_79 +1 javac 2m 52s the patch passed -1 checkstyle 0m 25s Patch generated 9 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 381, now 384). +1 mvneclipse 1m 9s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 45s the patch passed +1 javadoc 1m 43s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 47s the patch passed with JDK v1.7.0_79 -1 unit 10m 7s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66. +1 unit 0m 47s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66. -1 unit 2m 4s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66. -1 unit 121m 0s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_66. +1 unit 10m 19s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79. +1 unit 0m 58s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79. -1 unit 2m 12s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 unit 0m 19s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 24s Patch generated 1 ASF License warnings. 185m 33s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager   hadoop.mapreduce.v2.TestUberAM   hadoop.mapreduce.v2.TestMRJobsWithProfiler   hadoop.mapred.TestMRTimelineEventHandling JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.mapreduce.v2.app.TestMRAppComponentDependencies   org.apache.hadoop.mapreduce.TestLargeSort JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-05 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770757/0009-MAPREDUCE-5870.patch JIRA Issue MAPREDUCE-5870 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 47da65576f39 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-e8bd3ad/precommit/personality/hadoop.sh git revision trunk / 5667129 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Test case of TestUberAM is passing locally and also its fine for JDK 1.7. But Jenkins reports failure for JDK 1.8. I will check jenkins again to see the results.

          Show
          sunilg Sunil G added a comment - Test case of TestUberAM is passing locally and also its fine for JDK 1.7. But Jenkins reports failure for JDK 1.8. I will check jenkins again to see the results.
          Hide
          sunilg Sunil G added a comment -

          Hi Jason Lowe
          I have updated the patch correcting test failures, I will make changes for LocalJobRunner in separate ticket as you suggested earlier since it was already throwing exception.
          There were some failures in JDK 1.8, but locally it looks fine. I will re trigger jenkins again.

          Show
          sunilg Sunil G added a comment - Hi Jason Lowe I have updated the patch correcting test failures, I will make changes for LocalJobRunner in separate ticket as you suggested earlier since it was already throwing exception. There were some failures in JDK 1.8, but locally it looks fine. I will re trigger jenkins again.
          Hide
          jlowe Jason Lowe added a comment -

          I still think the TestUberAM failure is related to this patch. From the latest Jenkins run log at https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt

          testJobWithChangePriority(org.apache.hadoop.mapreduce.v2.TestUberAM)  Time elapsed: 21.508 sec  <<< FAILURE!
          java.lang.AssertionError: expected:<DEFAULT> but was:<HIGH>
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.junit.Assert.failNotEquals(Assert.java:743)
          	at org.junit.Assert.assertEquals(Assert.java:118)
          	at org.junit.Assert.assertEquals(Assert.java:144)
          	at org.apache.hadoop.mapreduce.v2.TestMRJobs.testJobWithChangePriority(TestMRJobs.java:276)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:497)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
          	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
          

          The failing test was added with this patch. Since it passes on JDK7, maybe the test has a race condition?

          Show
          jlowe Jason Lowe added a comment - I still think the TestUberAM failure is related to this patch. From the latest Jenkins run log at https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6118/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt testJobWithChangePriority(org.apache.hadoop.mapreduce.v2.TestUberAM) Time elapsed: 21.508 sec <<< FAILURE! java.lang.AssertionError: expected:<DEFAULT> but was:<HIGH> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.hadoop.mapreduce.v2.TestMRJobs.testJobWithChangePriority(TestMRJobs.java:276) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74) The failing test was added with this patch. Since it passes on JDK7, maybe the test has a race condition?
          Hide
          sunilg Sunil G added a comment -

          Thanks Jason Lowe.
          Yes, it's a race condition. Job priority change is done from client, then it's getting updated in RM. Once AM heartbeat is up, latest priority is updated to AM. Then with a client JobStatus query, client will get this. So a sleep is needed at client side till this cycle is done, which keeps varying. I ll look into the timeout again and will tune better in test case.

          Show
          sunilg Sunil G added a comment - Thanks Jason Lowe . Yes, it's a race condition. Job priority change is done from client, then it's getting updated in RM. Once AM heartbeat is up, latest priority is updated to AM. Then with a client JobStatus query, client will get this. So a sleep is needed at client side till this cycle is done, which keeps varying. I ll look into the timeout again and will tune better in test case.
          Hide
          sunilg Sunil G added a comment -

          Hi Jason Lowe
          I relooked on the test failures in TestUberRM and found that LocalContainerAllocator also need to update priority to JobImpl and it was missing. Added that change along with this ticket. Attaching patch for same.

          Show
          sunilg Sunil G added a comment - Hi Jason Lowe I relooked on the test failures in TestUberRM and found that LocalContainerAllocator also need to update priority to JobImpl and it was missing. Added that change along with this ticket. Attaching patch for same.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 7s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
          +1 mvninstall 3m 8s trunk passed
          +1 compile 2m 26s trunk passed with JDK v1.8.0_60
          +1 compile 2m 28s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 1m 44s trunk passed
          +1 mvneclipse 0m 56s trunk passed
          +1 findbugs 3m 17s trunk passed
          +1 javadoc 1m 16s trunk passed with JDK v1.8.0_60
          +1 javadoc 1m 26s trunk passed with JDK v1.7.0_79
          +1 mvninstall 1m 37s the patch passed
          +1 compile 2m 24s the patch passed with JDK v1.8.0_60
          +1 javac 2m 24s the patch passed
          +1 compile 2m 26s the patch passed with JDK v1.7.0_79
          +1 javac 2m 25s the patch passed
          -1 checkstyle 0m 20s Patch generated 9 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 385, now 388).
          +1 mvnsite 1m 48s the patch passed
          +1 mvneclipse 0m 56s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 3m 59s the patch passed
          +1 javadoc 1m 15s the patch passed with JDK v1.8.0_60
          +1 javadoc 1m 26s the patch passed with JDK v1.7.0_79
          -1 unit 9m 45s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60.
          +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_60.
          +1 unit 1m 47s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_60.
          -1 unit 106m 1s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_60.
          -1 unit 10m 26s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79.
          +1 unit 1m 0s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79.
          +1 unit 2m 17s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_79.
          -1 unit 129m 16s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 48s Patch generated 1 ASF License warnings.
          297m 15s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.local.TestLocalContainerAllocator
            hadoop.mapreduce.v2.TestMRJobsWithProfiler
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.local.TestLocalContainerAllocator
          JDK v1.7.0_79 Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-16
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12772521/0010-MAPREDUCE-5870.patch
          JIRA Issue MAPREDUCE-5870
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 74d3e0d54ebc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh
          git revision trunk / 855d529
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 230MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 3m 8s trunk passed +1 compile 2m 26s trunk passed with JDK v1.8.0_60 +1 compile 2m 28s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 21s trunk passed +1 mvnsite 1m 44s trunk passed +1 mvneclipse 0m 56s trunk passed +1 findbugs 3m 17s trunk passed +1 javadoc 1m 16s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 26s trunk passed with JDK v1.7.0_79 +1 mvninstall 1m 37s the patch passed +1 compile 2m 24s the patch passed with JDK v1.8.0_60 +1 javac 2m 24s the patch passed +1 compile 2m 26s the patch passed with JDK v1.7.0_79 +1 javac 2m 25s the patch passed -1 checkstyle 0m 20s Patch generated 9 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 385, now 388). +1 mvnsite 1m 48s the patch passed +1 mvneclipse 0m 56s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 59s the patch passed +1 javadoc 1m 15s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 26s the patch passed with JDK v1.7.0_79 -1 unit 9m 45s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60. +1 unit 0m 42s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_60. +1 unit 1m 47s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_60. -1 unit 106m 1s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_60. -1 unit 10m 26s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79. +1 unit 1m 0s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_79. +1 unit 2m 17s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_79. -1 unit 129m 16s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 48s Patch generated 1 ASF License warnings. 297m 15s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.local.TestLocalContainerAllocator   hadoop.mapreduce.v2.TestMRJobsWithProfiler JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.local.TestLocalContainerAllocator JDK v1.7.0_79 Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-16 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12772521/0010-MAPREDUCE-5870.patch JIRA Issue MAPREDUCE-5870 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 74d3e0d54ebc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-fa12328/precommit/personality/hadoop.sh git revision trunk / 855d529 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 230MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6166/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          TestLocalContainerAllocator are related. Adding a fix to handle failures.

          Show
          sunilg Sunil G added a comment - TestLocalContainerAllocator are related. Adding a fix to handle failures.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          +1 mvninstall 3m 17s trunk passed
          +1 compile 2m 36s trunk passed with JDK v1.8.0_66
          +1 compile 2m 38s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 1m 55s trunk passed
          +1 mvneclipse 1m 0s trunk passed
          -1 findbugs 1m 19s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings.
          +1 javadoc 1m 32s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 38s trunk passed with JDK v1.7.0_85
          +1 mvninstall 1m 43s the patch passed
          +1 compile 2m 40s the patch passed with JDK v1.8.0_66
          +1 javac 2m 40s the patch passed
          +1 compile 2m 40s the patch passed with JDK v1.7.0_85
          +1 javac 2m 40s the patch passed
          -1 checkstyle 0m 23s Patch generated 9 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 385, now 388).
          +1 mvnsite 1m 57s the patch passed
          +1 mvneclipse 1m 2s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 4m 19s the patch passed
          +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 37s the patch passed with JDK v1.7.0_85
          -1 unit 10m 6s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66.
          +1 unit 0m 49s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66.
          +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66.
          -1 unit 110m 20s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_66.
          -1 unit 10m 46s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_85.
          +1 unit 0m 59s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_85.
          +1 unit 2m 22s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_85.
          +1 unit 111m 28s hadoop-mapreduce-client-jobclient in the patch passed with JDK v1.7.0_85.
          -1 asflicense 0m 37s Patch generated 1 ASF License warnings.
          287m 48s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.v2.TestMRJobsWithProfiler
          JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.v2.app.job.impl.TestJobImpl
            hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:date2015-11-17
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12772639/0011-MAPREDUCE-5870.patch
          JIRA Issue MAPREDUCE-5870
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6d47915fbc08 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-4659377/precommit/personality/hadoop.sh
          git revision trunk / 6502d59
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 228MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. +1 mvninstall 3m 17s trunk passed +1 compile 2m 36s trunk passed with JDK v1.8.0_66 +1 compile 2m 38s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 22s trunk passed +1 mvnsite 1m 55s trunk passed +1 mvneclipse 1m 0s trunk passed -1 findbugs 1m 19s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings. +1 javadoc 1m 32s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 38s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 43s the patch passed +1 compile 2m 40s the patch passed with JDK v1.8.0_66 +1 javac 2m 40s the patch passed +1 compile 2m 40s the patch passed with JDK v1.7.0_85 +1 javac 2m 40s the patch passed -1 checkstyle 0m 23s Patch generated 9 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 385, now 388). +1 mvnsite 1m 57s the patch passed +1 mvneclipse 1m 2s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 19s the patch passed +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 37s the patch passed with JDK v1.7.0_85 -1 unit 10m 6s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66. +1 unit 0m 49s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66. +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66. -1 unit 110m 20s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_66. -1 unit 10m 46s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_85. +1 unit 0m 59s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_85. +1 unit 2m 22s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_85. +1 unit 111m 28s hadoop-mapreduce-client-jobclient in the patch passed with JDK v1.7.0_85. -1 asflicense 0m 37s Patch generated 1 ASF License warnings. 287m 48s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.v2.TestMRJobsWithProfiler JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.v2.app.job.impl.TestJobImpl   hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt Subsystem Report/Notes Docker Image:yetus/hadoop:date2015-11-17 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12772639/0011-MAPREDUCE-5870.patch JIRA Issue MAPREDUCE-5870 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6d47915fbc08 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-4659377/precommit/personality/hadoop.sh git revision trunk / 6502d59 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 228MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6168/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Test case failures are not related. TestJobImpl is passing locally. Others are failing w/o the patch also. I will see whether any test failure tickets are raised, if not will do the same.

          Show
          sunilg Sunil G added a comment - Test case failures are not related. TestJobImpl is passing locally. Others are failing w/o the patch also. I will see whether any test failure tickets are raised, if not will do the same.
          Hide
          jlowe Jason Lowe added a comment -

          It would be nice to cleanup most of the checkstyle nits, as there are a lot of switch statements without default clauses that were added. I realize semantically we're covered since the fall-through case returns some default value or throws, but we can trivially remove the warning by having an explicit default clause that explicitly breaks or move the fallthrough code into that default case.

          Other than that latest patch looks good to me.

          Show
          jlowe Jason Lowe added a comment - It would be nice to cleanup most of the checkstyle nits, as there are a lot of switch statements without default clauses that were added. I realize semantically we're covered since the fall-through case returns some default value or throws, but we can trivially remove the warning by having an explicit default clause that explicitly breaks or move the fallthrough code into that default case. Other than that latest patch looks good to me.
          Hide
          sunilg Sunil G added a comment -

          Sure Jason Lowe. I will upload a patch addressing the same.

          Show
          sunilg Sunil G added a comment - Sure Jason Lowe . I will upload a patch addressing the same.
          Hide
          sunilg Sunil G added a comment -

          Uploading new set of patch addressing the checkstyle comments.

          Show
          sunilg Sunil G added a comment - Uploading new set of patch addressing the checkstyle comments.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          +1 mvninstall 10m 13s trunk passed
          +1 compile 4m 32s trunk passed with JDK v1.8.0_66
          +1 compile 3m 43s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 24s trunk passed
          +1 mvnsite 2m 16s trunk passed
          +1 mvneclipse 1m 6s trunk passed
          -1 findbugs 1m 25s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings.
          +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 34s trunk passed with JDK v1.7.0_85
          +1 mvninstall 1m 54s the patch passed
          +1 compile 3m 24s the patch passed with JDK v1.8.0_66
          -1 javac 9m 2s hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66 with JDK v1.8.0_66 generated 21 new issues (was 360, now 360).
          +1 javac 3m 24s the patch passed
          +1 compile 3m 30s the patch passed with JDK v1.7.0_85
          -1 javac 12m 32s hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85 with JDK v1.7.0_85 generated 21 new issues (was 365, now 365).
          +1 javac 3m 30s the patch passed
          -1 checkstyle 0m 22s Patch generated 3 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 385, now 381).
          +1 mvnsite 2m 6s the patch passed
          +1 mvneclipse 1m 0s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 4m 16s the patch passed
          +1 javadoc 1m 22s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 33s the patch passed with JDK v1.7.0_85
          -1 unit 10m 2s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66.
          +1 unit 0m 54s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66.
          +1 unit 2m 12s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66.
          -1 unit 129m 23s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_66.
          -1 unit 10m 41s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_85.
          +1 unit 1m 1s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_85.
          +1 unit 2m 26s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_85.
          -1 unit 128m 14s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 34s Patch generated 1 ASF License warnings.
          336m 11s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.v2.TestMRJobsWithProfiler
          JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl
          JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
          JDK v1.7.0_85 Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:date2015-11-20
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773551/0012-MAPREDUCE-5870.patch
          JIRA Issue MAPREDUCE-5870
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 1848ecb4c024 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh
          git revision trunk / 4539131
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html
          javac hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66.txt
          javac hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 77MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. +1 mvninstall 10m 13s trunk passed +1 compile 4m 32s trunk passed with JDK v1.8.0_66 +1 compile 3m 43s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 24s trunk passed +1 mvnsite 2m 16s trunk passed +1 mvneclipse 1m 6s trunk passed -1 findbugs 1m 25s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings. +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 34s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 54s the patch passed +1 compile 3m 24s the patch passed with JDK v1.8.0_66 -1 javac 9m 2s hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66 with JDK v1.8.0_66 generated 21 new issues (was 360, now 360). +1 javac 3m 24s the patch passed +1 compile 3m 30s the patch passed with JDK v1.7.0_85 -1 javac 12m 32s hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85 with JDK v1.7.0_85 generated 21 new issues (was 365, now 365). +1 javac 3m 30s the patch passed -1 checkstyle 0m 22s Patch generated 3 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 385, now 381). +1 mvnsite 2m 6s the patch passed +1 mvneclipse 1m 0s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 16s the patch passed +1 javadoc 1m 22s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 33s the patch passed with JDK v1.7.0_85 -1 unit 10m 2s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_66. +1 unit 0m 54s hadoop-mapreduce-client-common in the patch passed with JDK v1.8.0_66. +1 unit 2m 12s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66. -1 unit 129m 23s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.8.0_66. -1 unit 10m 41s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_85. +1 unit 1m 1s hadoop-mapreduce-client-common in the patch passed with JDK v1.7.0_85. +1 unit 2m 26s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_85. -1 unit 128m 14s hadoop-mapreduce-client-jobclient in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 34s Patch generated 1 ASF License warnings. 336m 11s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.v2.TestMRJobsWithProfiler JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster JDK v1.7.0_85 Timed out junit tests org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl Subsystem Report/Notes Docker Image:yetus/hadoop:date2015-11-20 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773551/0012-MAPREDUCE-5870.patch JIRA Issue MAPREDUCE-5870 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1848ecb4c024 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh git revision trunk / 4539131 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html javac hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66.txt javac hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 77MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6177/console This message was automatically generated.
          Hide
          sunilg Sunil G added a comment -

          Findbugs warnings are unrelated. Few javac warnings are populated due to deprecation which is not introduced by this one. I will check why this happened.

          DistributedCache in org.apache.hadoop.mapreduce.filecache has been deprecated
          

          I will file a ticket for TestMRAppMaster.testMRAppMasterMidLock failure.

          Show
          sunilg Sunil G added a comment - Findbugs warnings are unrelated. Few javac warnings are populated due to deprecation which is not introduced by this one. I will check why this happened. DistributedCache in org.apache.hadoop.mapreduce.filecache has been deprecated I will file a ticket for TestMRAppMaster.testMRAppMasterMidLock failure.
          Hide
          sunilg Sunil G added a comment -

          MAPREDUCE-6555 is filed by Varun for tracking TestMRAppMaster failures.

          Show
          sunilg Sunil G added a comment - MAPREDUCE-6555 is filed by Varun for tracking TestMRAppMaster failures.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for updating the patch, Sunil! +1, agree that unit test failures and java warnings are unrelated to this patch. Committing this.

          Show
          jlowe Jason Lowe added a comment - Thanks for updating the patch, Sunil! +1, agree that unit test failures and java warnings are unrelated to this patch. Committing this.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks to Sunil for the contribution and to Devaraj, Jian, and Eric for additional review! I committed this to trunk and branch-2.

          Show
          jlowe Jason Lowe added a comment - Thanks to Sunil for the contribution and to Devaraj, Jian, and Eric for additional review! I committed this to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8881 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8881/)
          MAPREDUCE-5870. Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8881 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8881/ ) MAPREDUCE-5870 . Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #718 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/718/)
          MAPREDUCE-5870. Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #718 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/718/ ) MAPREDUCE-5870 . Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1450 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1450/)
          MAPREDUCE-5870. Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1450 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1450/ ) MAPREDUCE-5870 . Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #728 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/728/)
          MAPREDUCE-5870. Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #728 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/728/ ) MAPREDUCE-5870 . Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2660 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2660/)
          MAPREDUCE-5870. Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2660 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2660/ ) MAPREDUCE-5870 . Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          Hide
          sunilg Sunil G added a comment -

          Thank you very much Jason Lowe for the reviews, commit and the support. And thank you Jian He, Devaraj K and Eric Payne for the reviews !

          Show
          sunilg Sunil G added a comment - Thank you very much Jason Lowe for the reviews, commit and the support. And thank you Jian He , Devaraj K and Eric Payne for the reviews !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2577 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2577/)
          MAPREDUCE-5870. Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2577 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2577/ ) MAPREDUCE-5870 . Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #640 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/640/)
          MAPREDUCE-5870. Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #640 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/640/ ) MAPREDUCE-5870 . Support for passing Job priority through Application (jlowe: rev f634505d48d97e4d461980d68a0cbdf87223646d) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobConf.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobStatus.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/local/TestLocalContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJob.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobPriority.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java

            People

            • Assignee:
              sunilg Sunil G
              Reporter:
              sunilg Sunil G
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development