Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7278

Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 3.4.0
    • Fix Version/s: 3.3.0, 3.2.2, 3.4.0
    • Component/s: task
    • Labels:
      None

      Description

      When a failed task attempt container is stuck in FAIL_FINISHING_CONTAINER state for some time, we observe two task attempts are launched simultaneously even when speculative execution is disabled.

      This results in the below message shown in the killed attempts, indicating speculation has occurred. This is an issue for jobs which require speculative execution to be strictly disabled.

       

       

       

        Attachments

        1. MAPREDUCE-7278.001.patch
          3 kB
          Tarun Parimi
        2. MAPREDUCE-7278.002.patch
          3 kB
          Tarun Parimi
        3. MAPREDUCE-7278.003.patch
          8 kB
          Tarun Parimi
        4. MAPREDUCE-7278.004.patch
          9 kB
          Tarun Parimi
        5. Screen Shot 2020-04-30 at 8.04.27 PM.png
          40 kB
          Tarun Parimi

          Issue Links

            Activity

              People

              • Assignee:
                tarunparimi Tarun Parimi
                Reporter:
                tarunparimi Tarun Parimi
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: