Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7169

Speculative attempts should not run on the same node

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.7.2
    • None
    • yarn
    • None

    Description

      I found in all versions of yarn, Speculative Execution may set the speculative task to the node of original task.What i have read is only it will try to have one more task attempt. haven't seen any place mentioning not on same node.It is unreasonable.If the node have some problems lead to tasks execution will be very slow. and then placement the speculative task to same node cannot help the problematic task.
      In our cluster （version 2.7.2，2700 nodes），this phenomenon appear almost everyday.

      Attachments

        1. MAPREDUCE-7169.007.patch
          37 kB
          Bilwa S T
        2. MAPREDUCE-7169.006.patch
          37 kB
          Bilwa S T
        3. MAPREDUCE-7169.005.patch
          37 kB
          Bilwa S T
        4. MAPREDUCE-7169.004.patch
          37 kB
          Bilwa S T
        5. MAPREDUCE-7169-003.patch
          33 kB
          Bilwa S T
        6. MAPREDUCE-7169-002.patch
          21 kB
          Bilwa S T
        7. MAPREDUCE-7169-001.patch
          31 kB
          Bilwa S T
        8. image-2018-12-03-09-54-07-859.png
          72 kB
          Lee chen

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6066 Speculative attempts should not run on the same node as their original attempt

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              BilwaST Bilwa S T
              lichen1109 Lee chen
              Votes:
              1 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated: