Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4257

Support fair-sharing option within a MR2 Capacity Scheduler queue

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: capacity-sched, mrv2
    • Labels:
      None

      Description

      The fair scheduler can run jobs in a single pool (queue) in FIFO or fair share mode. In FIFO mode one job runs at a time, in priority order, while in fair share mode multiple jobs can run at the same time, and they share the capacity of the pool. This JIRA is to add the latter feature to Capacity Scheduler as an option - the default would remain FIFO.

        Issue Links

          Activity

          Hide
          Karthik Kambatla added a comment -

          MR-4327 attempts DRF (fair-sharing) across multiple resources.

          That said, it might still be worthwhile to have MR-4257 for fairness based on a single resource (memory).

          Should we treat DRF as a super-set of 4327 or treat them as separate JIRAs?

          Show
          Karthik Kambatla added a comment - MR-4327 attempts DRF (fair-sharing) across multiple resources. That said, it might still be worthwhile to have MR-4257 for fairness based on a single resource (memory). Should we treat DRF as a super-set of 4327 or treat them as separate JIRAs?
          Hide
          Arun C Murthy added a comment -

          The CS already supports user-sharing. Should we close this?

          Show
          Arun C Murthy added a comment - The CS already supports user-sharing. Should we close this?
          Hide
          Tom White added a comment -

          Yes, I don't think we need this.

          Show
          Tom White added a comment - Yes, I don't think we need this.

            People

            • Assignee:
              Karthik Kambatla
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development