Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7177

Disable speculative execution in TestDFSIO

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0, 2.8.5
    • 2.10.0, 3.1.2, 2.8.6, 3.3.0, 3.2.1, 2.9.3
    • None
    • None
    • Reviewed

    Description

      When TestDFSIO runs in a certain environment where a subset of mappers are slow, the speculative execution can start. In the write phase, this will make existing mapper to fail in next addBlock() since the output files are overwritten.

      To make the benchmark more predictable and repeatable, speculation must be implicitly disabled in TestDFSIO itself.

      Attachments

        1. MAPREDUCE-7177.001.patch
          0.8 kB
          Zhaohui Xin

        Activity

          People

            uranus Zhaohui Xin
            kihwal Kihwal Lee
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: