Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2198

Allow FairScheduler to control the number of slots on each TaskTracker

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: contrib/fair-share
    • Labels:
      None

      Description

      We can set the number of slots on the TaskTracker to be high and let FairScheduler handles the slots.
      This approach allows us to change the number of slots on each node dynamically.
      The administrator can change the number of slots with a CLI tool.

      One use case of this is for upgrading the MapReduce.
      Instead of restarting the cluster, we can run the new MapReduce on the same cluster.
      And use the CLI tool to gradually migrate the slots.
      This way we don't lost the progress fo the jobs that's already executed.

      1. MAPREDUCE-2198-v2.txt
        31 kB
        Scott Chen
      2. MAPREDUCE-2198.txt
        28 kB
        Scott Chen

        Issue Links

          Activity

          Scott Chen created issue -
          Scott Chen made changes -
          Field Original Value New Value
          Link This issue relates to MAPREDUCE-2108 [ MAPREDUCE-2108 ]
          Scott Chen made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Scott Chen made changes -
          Attachment MAPREDUCE-2198.txt [ 12460482 ]
          Scott Chen made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Scott Chen made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Scott Chen made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Scott Chen made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.23.0 [ 12315570 ]
          Affects Version/s 0.22.0 [ 12314184 ]
          Fix Version/s 0.23.0 [ 12315570 ]
          Fix Version/s 0.22.0 [ 12314184 ]
          Scott Chen made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Scott Chen made changes -
          Attachment MAPREDUCE-2198.txt [ 12460484 ]
          Scott Chen made changes -
          Attachment MAPREDUCE-2198.txt [ 12460482 ]
          Scott Chen made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Scott Chen made changes -
          Attachment MAPREDUCE-2198-v2.txt [ 12464776 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Scott Chen made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Scott Chen
              Reporter:
              Scott Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development