Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: mrv2, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Helps in improving app performance. RM should be able to handle this, as the heartbeats aren't really costly.

        Activity

        Hide
        Hitesh Shah added a comment -

        Trivial patch based on findings from performance benchmarks.

        Show
        Hitesh Shah added a comment - Trivial patch based on findings from performance benchmarks.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12511952/MR-3718.1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.mapred.TestJobClientGetJob
        org.apache.hadoop.mapred.TestMRWithDistributedCache
        org.apache.hadoop.mapred.TestLocalModeWithNewApis

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1680//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1680//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511952/MR-3718.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestJobClientGetJob org.apache.hadoop.mapred.TestMRWithDistributedCache org.apache.hadoop.mapred.TestLocalModeWithNewApis +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1680//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1680//console This message is automatically generated.
        Hide
        Hitesh Shah added a comment -

        Test failures seem to be a recurring ( but not always reproducible ) theme in previous pre-commit builds too. Not related to this patch. Will file a separate jira for the jenkin test failures.

        Show
        Hitesh Shah added a comment - Test failures seem to be a recurring ( but not always reproducible ) theme in previous pre-commit builds too. Not related to this patch. Will file a separate jira for the jenkin test failures.
        Hide
        Siddharth Seth added a comment -

        +1. Patch looks good. Unit test failures aren't related.

        Show
        Siddharth Seth added a comment - +1. Patch looks good. Unit test failures aren't related.
        Hide
        Siddharth Seth added a comment -

        Committed to trunk and branch-0.23. Thanks Hitesh.

        Show
        Siddharth Seth added a comment - Committed to trunk and branch-0.23. Thanks Hitesh.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1667 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1667/)
        MAPREDUCE-3718. Amending incorrect JIRA number in CHANGES.txt

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1667 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1667/ ) MAPREDUCE-3718 . Amending incorrect JIRA number in CHANGES.txt sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1595 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1595/)
        MAPREDUCE-3718. Amending incorrect JIRA number in CHANGES.txt

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1595 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1595/ ) MAPREDUCE-3718 . Amending incorrect JIRA number in CHANGES.txt sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #418 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/418/)
        merge MAPREDUCE-3718 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #418 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/418/ ) merge MAPREDUCE-3718 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #427 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/427/)
        merge MAPREDUCE-3718 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #427 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/427/ ) merge MAPREDUCE-3718 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1611 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1611/)
        MAPREDUCE-3718. Amending incorrect JIRA number in CHANGES.txt

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1611 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1611/ ) MAPREDUCE-3718 . Amending incorrect JIRA number in CHANGES.txt sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #443 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/443/)
        merge MAPREDUCE-3718 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #443 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/443/ ) merge MAPREDUCE-3718 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #938 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/938/)
        MAPREDUCE-3718. Amending incorrect JIRA number in CHANGES.txt

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #938 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/938/ ) MAPREDUCE-3718 . Amending incorrect JIRA number in CHANGES.txt sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #151 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/151/)
        merge MAPREDUCE-3718 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #151 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/151/ ) merge MAPREDUCE-3718 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #971 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/971/)
        MAPREDUCE-3718. Amending incorrect JIRA number in CHANGES.txt

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #971 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/971/ ) MAPREDUCE-3718 . Amending incorrect JIRA number in CHANGES.txt sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236358 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #173 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/173/)
        merge MAPREDUCE-3718 from trunk

        sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #173 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/173/ ) merge MAPREDUCE-3718 from trunk sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236360 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java

          People

          • Assignee:
            Hitesh Shah
            Reporter:
            Vinod Kumar Vavilapalli
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development