Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7278

Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 3.4.0
    • Fix Version/s: 3.3.0, 3.2.2, 3.4.0
    • Component/s: task
    • Labels:
      None

      Description

      When a failed task attempt container is stuck in FAIL_FINISHING_CONTAINER state for some time, we observe two task attempts are launched simultaneously even when speculative execution is disabled.

      This results in the below message shown in the killed attempts, indicating speculation has occurred. This is an issue for jobs which require speculative execution to be strictly disabled.

       

       

       

        Attachments

        1. MAPREDUCE-7278.001.patch
          3 kB
          Tarun Parimi
        2. MAPREDUCE-7278.002.patch
          3 kB
          Tarun Parimi
        3. MAPREDUCE-7278.003.patch
          8 kB
          Tarun Parimi
        4. MAPREDUCE-7278.004.patch
          9 kB
          Tarun Parimi
        5. Screen Shot 2020-04-30 at 8.04.27 PM.png
          40 kB
          Tarun Parimi

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              tarunparimi Tarun Parimi Assign to me
              Reporter:
              tarunparimi Tarun Parimi

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment