Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: contrib/gridmix
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Adds High-Ram feature emulation in Gridmix.
    • Tags:
      gridmix high-ram

      Description

      Gridmix currently ignores high ram job configuration of the original job. It would be nice if Gridmix configures the simulated job's high ram parameters such that the simulated job has same effect on the job scheduler & task-tracker as the original job.

      1. mapreduce-2543-v1.4.patch
        16 kB
        Amar Kamat
      2. mapreduce-2543-v1.2.patch
        14 kB
        Amar Kamat

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #722 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/722/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #722 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/722/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #710 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/710/)
        MAPREDUCE-2543. [Gridmix] High-Ram feature emulation testcase. (amarrk)

        amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1130554
        Files :

        • /hadoop/mapreduce/trunk/CHANGES.txt
        • /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestHighRamJob.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #710 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/710/ ) MAPREDUCE-2543 . [Gridmix] High-Ram feature emulation testcase. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1130554 Files : /hadoop/mapreduce/trunk/CHANGES.txt /hadoop/mapreduce/trunk/src/contrib/gridmix/src/test/org/apache/hadoop/mapred/gridmix/TestHighRamJob.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #709 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/709/)
        MAPREDUCE-2543. [Gridmix] High-Ram feature emulation in Gridmix. (amarrk)

        amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1130550
        Files :

        • /hadoop/mapreduce/trunk/CHANGES.txt
        • /hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/gridmix.xml
        • /hadoop/mapreduce/trunk/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #709 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/709/ ) MAPREDUCE-2543 . [Gridmix] High-Ram feature emulation in Gridmix. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1130550 Files : /hadoop/mapreduce/trunk/CHANGES.txt /hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/gridmix.xml /hadoop/mapreduce/trunk/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java
        Hide
        Amar Kamat added a comment -

        I just committed the latest patch to trunk. Thanks Ravi for the review.

        Show
        Amar Kamat added a comment - I just committed the latest patch to trunk. Thanks Ravi for the review.
        Hide
        Ravi Gummadi added a comment -

        Patch looks fine to me.
        +1

        Show
        Ravi Gummadi added a comment - Patch looks fine to me. +1
        Hide
        Amar Kamat added a comment -

        TestMRCLI failure is not related to this patch. This patch was QAed by Vinay and reviewed (offline) by Ravi.

        Show
        Amar Kamat added a comment - TestMRCLI failure is not related to this patch. This patch was QAed by Vinay and reviewed (offline) by Ravi.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12481211/mapreduce-2543-v1.4.patch
        against trunk revision 1130393.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:
        org.apache.hadoop.cli.TestMRCLI

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/333//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/333//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/333//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481211/mapreduce-2543-v1.4.patch against trunk revision 1130393. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestMRCLI +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/333//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/333//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/333//console This message is automatically generated.
        Hide
        Amar Kamat added a comment -

        Attaching a new patch fixing a bug found by Vinay. Incorporated Ravi's offline comments. test-patch and Gridmix unit tests passed.

        Show
        Amar Kamat added a comment - Attaching a new patch fixing a bug found by Vinay. Incorporated Ravi's offline comments. test-patch and Gridmix unit tests passed.
        Hide
        Amar Kamat added a comment -

        Attaching a patch that adds support for High Ram jobs in Gridmix. Added a unit test-case to test the same. test-patch and test-contrib passed on my box.

        Show
        Amar Kamat added a comment - Attaching a patch that adds support for High Ram jobs in Gridmix. Added a unit test-case to test the same. test-patch and test-contrib passed on my box.

          People

          • Assignee:
            Amar Kamat
            Reporter:
            Amar Kamat
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development