Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5132

Enhance FairScheduler queue selection with Unix-SubGroups

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      I have clients that use the primary group for other reasons and would like to select the fairScheduler queue by subgroup.

      Currently the fairscheduler can only link a job config property to a fairscheduler queue and subgroups are not in the job config.

      Now thankfully we can get this data from UserGroupInformation.createProxyUser which is cool because this is how TT currently see if it can copy data when kerberos is enabled. Once we have the sub-groups we just connect the first sub-group that matches a queue.

      I'll put up some code that I used to solve the problem in the short term.

      Note that there is an issue with this code. It doesn't populate the TT Web UI drop down menu correctly. I think I have a solution for that, but before I implement that I was wondering if the community has any feedback first.

      1. FairSchedulerPlusCode.zip
        5 kB
        Ted Malaska
      2. MAPREDUCE-5132.NotTested.patch
        9 kB
        Ted Malaska

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ted Malaska
          • Votes:
            2 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:

              Development