Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6912

CompletedMapTask contains failed and killed will lead mapreduce.job.reduce.slowstart.completedmaps not work properly

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.6.2
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      if the conf is set to 1, is about to make reduce task to be scheduled only after all map task success

      int completedMapsForReduceSlowstart = (int)Math.ceil(reduceSlowStart *
      totalMaps);
      if(completedMaps < completedMapsForReduceSlowstart) {}
      
      ///////////////////////////
      
      int completedMaps = getJob().getCompletedMaps();
      
      ///////////////////////////
      JobImpl
       @Override
      public int getCompletedMaps() {
      readLock.lock();
      try {
      return succeededMapTaskCount + failedMapTaskCount + killedMapTaskCount;
      } finally {
      readLock.unlock();
      }
      }
      

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:

                Issue deployment