Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1635

ResourceEstimator does not work after MAPREDUCE-842

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed a bug related to resource estimation for disk-based scheduling by modifying TaskTracker to return correct map output size for the completed maps and -1 for other tasks or failures.

      Description

      MAPREDUCE-842 changed Child's mapred.local.dir to have attemptDir as the base local directory. Also assumption is that
      org.apache.hadoop.mapred.MapOutputFile always gets Child's mapred.local.dir.
      But, MapOuptutFile.getOutputFile() is called from TaskTracker's conf, which does not find the output file. Thus TaskTracker.tryToGetOutputSize() always returns -1.

      1. patch-1635.txt
        8 kB
        Amareshwari Sriramadasu
      2. patch-1635-1.txt
        9 kB
        Amareshwari Sriramadasu
      3. ASF.LICENSE.NOT.GRANTED--patch-1635-ydist.txt
        9 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development