Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1761

FairScheduler should allow separate configuration of node and rack locality wait time

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It would be nice that we can separately assign rack locality wait time.
      In our use case, we would set node locality wait to zero and wait only rack locality.

      I propose that we add two parameters
      mapred.fairscheduler.locality.delay.nodetorack
      mapred.fairscheduler.locality.delay.racktoany
      This allows specifying the wait time on each stage.

      And we can use
      mapred.fairscheduler.locality.delay
      as the default value of the above fields so that this is backward compatible.

      Thoughts?

        Attachments

        1. MAPREDUCE-1761-v1.2.txt
          6 kB
          Scott Chen
        2. MAPREDUCE-1761-v1.1.txt
          6 kB
          Scott Chen
        3. MAPREDUCE-1761.txt
          6 kB
          Scott Chen

          Activity

            People

            • Assignee:
              schen Scott Chen
              Reporter:
              schen Scott Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: