Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5132

Enhance FairScheduler queue selection with Unix-SubGroups

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      I have clients that use the primary group for other reasons and would like to select the fairScheduler queue by subgroup.

      Currently the fairscheduler can only link a job config property to a fairscheduler queue and subgroups are not in the job config.

      Now thankfully we can get this data from UserGroupInformation.createProxyUser which is cool because this is how TT currently see if it can copy data when kerberos is enabled. Once we have the sub-groups we just connect the first sub-group that matches a queue.

      I'll put up some code that I used to solve the problem in the short term.

      Note that there is an issue with this code. It doesn't populate the TT Web UI drop down menu correctly. I think I have a solution for that, but before I implement that I was wondering if the community has any feedback first.

      Attachments

        1. MAPREDUCE-5132.NotTested.patch
          9 kB
          Theodore michael Malaska
        2. FairSchedulerPlusCode.zip
          5 kB
          Theodore michael Malaska

        Activity

          People

            Unassigned Unassigned
            ted.m Theodore michael Malaska
            Votes:
            2 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated: