Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1635

ResourceEstimator does not work after MAPREDUCE-842

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed a bug related to resource estimation for disk-based scheduling by modifying TaskTracker to return correct map output size for the completed maps and -1 for other tasks or failures.

      Description

      MAPREDUCE-842 changed Child's mapred.local.dir to have attemptDir as the base local directory. Also assumption is that
      org.apache.hadoop.mapred.MapOutputFile always gets Child's mapred.local.dir.
      But, MapOuptutFile.getOutputFile() is called from TaskTracker's conf, which does not find the output file. Thus TaskTracker.tryToGetOutputSize() always returns -1.

        Attachments

        1. patch-1635-1.txt
          9 kB
          Amareshwari Sriramadasu
        2. patch-1635.txt
          8 kB
          Amareshwari Sriramadasu
        3. ASF.LICENSE.NOT.GRANTED--patch-1635-ydist.txt
          9 kB
          Amareshwari Sriramadasu

          Issue Links

            Activity

              People

              • Assignee:
                amareshwari Amareshwari Sriramadasu
                Reporter:
                amareshwari Amareshwari Sriramadasu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: