Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5132

Enhance FairScheduler queue selection with Unix-SubGroups

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      I have clients that use the primary group for other reasons and would like to select the fairScheduler queue by subgroup.

      Currently the fairscheduler can only link a job config property to a fairscheduler queue and subgroups are not in the job config.

      Now thankfully we can get this data from UserGroupInformation.createProxyUser which is cool because this is how TT currently see if it can copy data when kerberos is enabled. Once we have the sub-groups we just connect the first sub-group that matches a queue.

      I'll put up some code that I used to solve the problem in the short term.

      Note that there is an issue with this code. It doesn't populate the TT Web UI drop down menu correctly. I think I have a solution for that, but before I implement that I was wondering if the community has any feedback first.

        Attachments

        1. MAPREDUCE-5132.NotTested.patch
          9 kB
          Theodore michael Malaska
        2. FairSchedulerPlusCode.zip
          5 kB
          Theodore michael Malaska

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ted.m Theodore michael Malaska
            • Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated: