Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1367

LocalJobRunner should support parallel mapper execution

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The LocalJobRunner currently supports only a single execution thread. Given the prevalence of multi-core CPUs, it makes sense to allow users to run multiple tasks in parallel for improved performance on small (local-only) jobs.

      1. MAPREDUCE-1367.2.patch
        24 kB
        Aaron Kimball
      2. MAPREDUCE-1367.3.patch
        24 kB
        Aaron Kimball
      3. MAPREDUCE-1367.4.patch
        26 kB
        Aaron Kimball
      4. MAPREDUCE-1367.5.patch
        28 kB
        Aaron Kimball
      5. MAPREDUCE-1367.6.patch
        28 kB
        Aaron Kimball
      6. MAPREDUCE-1367.7.patch
        29 kB
        Aaron Kimball
      7. MAPREDUCE-1367.patch
        22 kB
        Aaron Kimball
      8. MAPREDUCE-1367-branch1.patch
        28 kB
        Sandy Ryza

        Issue Links

          Activity

          Hide
          Aaron Kimball added a comment -

          Attaching a patch that implements this improvement. This patch includes a test case which launches 6 mappers concurrently; these mappers run on a variety of schedules (some are faster, some are slower) in an attempt to suss out any race conditions that might develop.

          The level of parallelism is controlled by a new parameter: mapred.local.map.tasks.maximum. This defaults to 1, so that unspecified behavior is as before.

          I also tested this by running the 'pi' example from the command line:

          bin/hadoop jar hadoop-mapred-examples-0.22.0-SNAPSHOT.jar pi -D mapreduce.jobtracker.address=local -D mapreduce.local.map.tasks.maximum=2 20 5000000
          

          With mapreduce.local.map.tasks.maximum set to 1, this takes 13.5 seconds on my machine. With it set to 2 or above (I have two cores), the runtime drops to 8.5 seconds.

          Show
          Aaron Kimball added a comment - Attaching a patch that implements this improvement. This patch includes a test case which launches 6 mappers concurrently; these mappers run on a variety of schedules (some are faster, some are slower) in an attempt to suss out any race conditions that might develop. The level of parallelism is controlled by a new parameter: mapred.local.map.tasks.maximum . This defaults to 1, so that unspecified behavior is as before. I also tested this by running the 'pi' example from the command line: bin/hadoop jar hadoop-mapred-examples-0.22.0-SNAPSHOT.jar pi -D mapreduce.jobtracker.address=local -D mapreduce.local.map.tasks.maximum=2 20 5000000 With mapreduce.local.map.tasks.maximum set to 1, this takes 13.5 seconds on my machine. With it set to 2 or above (I have two cores), the runtime drops to 8.5 seconds.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12429800/MAPREDUCE-1367.patch
          against trunk revision 897118.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 2 new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429800/MAPREDUCE-1367.patch against trunk revision 897118. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/369/console This message is automatically generated.
          Hide
          Aaron Kimball added a comment -

          New patch; rewrites counter handling to make TestJobCounters pass.

          Show
          Aaron Kimball added a comment - New patch; rewrites counter handling to make TestJobCounters pass.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12429958/MAPREDUCE-1367.2.patch
          against trunk revision 898019.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 2 new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429958/MAPREDUCE-1367.2.patch against trunk revision 898019. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/261/console This message is automatically generated.
          Hide
          Aaron Kimball added a comment -

          Tests that failed in Hudson all pass locally. These tests don't appear to use the LocalJobRunner.

          Show
          Aaron Kimball added a comment - Tests that failed in Hudson all pass locally. These tests don't appear to use the LocalJobRunner.
          Hide
          Aaron Kimball added a comment -

          new patch addresses findbugs warnings.

          Show
          Aaron Kimball added a comment - new patch addresses findbugs warnings.
          Hide
          Chris Douglas added a comment -

          As a first pass:

          • map_tasks is volatile, but increment/decrement is not atomic. Consider AtomicInteger instead
          • getMapExecutor should be named createMapExecutor ; adding tasks to the ExecutorService belongs at the caller. The logging should probably be at DEBUG instead of INFO
          • storedException should probably be Throwable instead of Exception
          • if LOCAL_MAX_MAPS is < 1, that should be an error

          Otherwise, this looks pretty good.

          Show
          Chris Douglas added a comment - As a first pass: map_tasks is volatile, but increment/decrement is not atomic. Consider AtomicInteger instead getMapExecutor should be named createMapExecutor ; adding tasks to the ExecutorService belongs at the caller. The logging should probably be at DEBUG instead of INFO storedException should probably be Throwable instead of Exception if LOCAL_MAX_MAPS is < 1, that should be an error Otherwise, this looks pretty good.
          Hide
          Aaron Kimball added a comment -

          Chris, all those suggestions make sense. Here's a new patch that implements them.

          I also added a static getter/setter API for the config param

          public static void LocalJobRunner.setLocalMaxRunningMaps(JobContext, int)
          public static int LocalJobRunner.getLocalMaxRunningMaps(JobContext)
          
          Show
          Aaron Kimball added a comment - Chris, all those suggestions make sense. Here's a new patch that implements them. I also added a static getter/setter API for the config param public static void LocalJobRunner.setLocalMaxRunningMaps(JobContext, int ) public static int LocalJobRunner.getLocalMaxRunningMaps(JobContext)
          Hide
          Todd Lipcon added a comment -
          • MapTaskRunnable: make most of the instance variables final?
          • it should be clearer that mapOutputFiles is an "out param" for getMapTaskRunnables - ie that it's expected to be intitially empty. Alternatively, it looks like you may be able to remove this variable entirely by instead storing the MapOutputFile in the MapRunnable instance, and then iterating directly over the MapRunnables in the reducer. Does that make sense? However, the way you've done it is less invasive to the reduce side, so if you don't see the benefit, feel free to ignore this suggestion.
          • Does this handle the degenerate case of 0-map jobs? It sounds ridiculous, but I recall previous JIRAs for this situation, since occasionally people have a cron job that periodically processes a given directory. If the directory is empty, it may generate a job with no input splits and thus no tasks.

          Aside from that, looks good to me.

          Show
          Todd Lipcon added a comment - MapTaskRunnable: make most of the instance variables final? it should be clearer that mapOutputFiles is an "out param" for getMapTaskRunnables - ie that it's expected to be intitially empty. Alternatively, it looks like you may be able to remove this variable entirely by instead storing the MapOutputFile in the MapRunnable instance, and then iterating directly over the MapRunnables in the reducer. Does that make sense? However, the way you've done it is less invasive to the reduce side, so if you don't see the benefit, feel free to ignore this suggestion. Does this handle the degenerate case of 0-map jobs? It sounds ridiculous, but I recall previous JIRAs for this situation, since occasionally people have a cron job that periodically processes a given directory. If the directory is empty, it may generate a job with no input splits and thus no tasks. Aside from that, looks good to me.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12430043/MAPREDUCE-1367.4.patch
          against trunk revision 898486.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12430043/MAPREDUCE-1367.4.patch against trunk revision 898486. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/378/console This message is automatically generated.
          Hide
          Aaron Kimball added a comment -

          Todd, good catch: it didn't handle the 0-mapper case. It does now. Testcase included.

          Show
          Aaron Kimball added a comment - Todd, good catch: it didn't handle the 0-mapper case. It does now. Testcase included.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12430081/MAPREDUCE-1367.5.patch
          against trunk revision 898486.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12430081/MAPREDUCE-1367.5.patch against trunk revision 898486. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/268/console This message is automatically generated.
          Hide
          Jeff Hammerbacher added a comment -

          What is the delay in committing this patch?

          Show
          Jeff Hammerbacher added a comment - What is the delay in committing this patch?
          Hide
          Tom White added a comment -

          Aaron, this patch has gone stale. Could you regenerate, then I think it's ready to be committed.

          Show
          Tom White added a comment - Aaron, this patch has gone stale. Could you regenerate, then I think it's ready to be committed.
          Hide
          Aaron Kimball added a comment -

          new patch, sync'd with trunk.

          Show
          Aaron Kimball added a comment - new patch, sync'd with trunk.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12431509/MAPREDUCE-1367.6.patch
          against trunk revision 903508.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12431509/MAPREDUCE-1367.6.patch against trunk revision 903508. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/288/console This message is automatically generated.
          Hide
          Aaron Kimball added a comment -

          Test failures are all unrelated (TT blacklisting has now falied two tests in a row?)

          Show
          Aaron Kimball added a comment - Test failures are all unrelated (TT blacklisting has now falied two tests in a row?)
          Hide
          Aaron Kimball added a comment -

          new patch, sync'd with trunk.

          Show
          Aaron Kimball added a comment - new patch, sync'd with trunk.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12431823/MAPREDUCE-1367.7.patch
          against trunk revision 904609.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12431823/MAPREDUCE-1367.7.patch against trunk revision 904609. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/418/console This message is automatically generated.
          Hide
          Aaron Kimball added a comment -

          Test failures seem unrelated - classpath issues on hudson again?

          Show
          Aaron Kimball added a comment - Test failures seem unrelated - classpath issues on hudson again?
          Hide
          Tom White added a comment -

          I've just committed this. Thanks Aaron!

          Show
          Tom White added a comment - I've just committed this. Thanks Aaron!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #221 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/221/)
          . LocalJobRunner should support parallel mapper execution. Contributed by Aaron Kimball.

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #221 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/221/ ) . LocalJobRunner should support parallel mapper execution. Contributed by Aaron Kimball.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #225 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/225/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #225 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/225/ )
          Hide
          Ramkumar Vadali added a comment -

          @Aaron, Just curious, is this being used in production? If so, could you please outline the use case?

          Show
          Ramkumar Vadali added a comment - @Aaron, Just curious, is this being used in production? If so, could you please outline the use case?
          Hide
          Sandy Ryza added a comment -

          I submitted a patch for branch-1, as it was never backported.

          Show
          Sandy Ryza added a comment - I submitted a patch for branch-1, as it was never backported.
          Hide
          Sandy Ryza added a comment -

          Regenerated branch-1 patch

          Show
          Sandy Ryza added a comment - Regenerated branch-1 patch
          Hide
          Sandy Ryza added a comment -

          TestLocalRunner passes

          Show
          Sandy Ryza added a comment - TestLocalRunner passes
          Hide
          Johannes Zillmann added a comment -

          Hey guys, is 0.21.0 the only branch this is fixed in ?
          Not in 1.x, not in 2.x ?

          Show
          Johannes Zillmann added a comment - Hey guys, is 0.21.0 the only branch this is fixed in ? Not in 1.x, not in 2.x ?

            People

            • Assignee:
              Aaron Kimball
              Reporter:
              Aaron Kimball
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development