Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7177

Disable speculative execution in TestDFSIO

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0, 2.8.5
    • Fix Version/s: 2.10.0, 3.1.2, 2.8.6, 3.3.0, 3.2.1, 2.9.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When TestDFSIO runs in a certain environment where a subset of mappers are slow, the speculative execution can start. In the write phase, this will make existing mapper to fail in next addBlock() since the output files are overwritten.

      To make the benchmark more predictable and repeatable, speculation must be implicitly disabled in TestDFSIO itself.

        Attachments

        1. MAPREDUCE-7177.001.patch
          0.8 kB
          Zhaohui Xin

          Activity

            People

            • Assignee:
              uranus Zhaohui Xin
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: