Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8424

Support fair scheduler user queue mapping in non-impersonation mode

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.1.0
    • Shims

    Description

      Under non-impersonation mode, all MR jobs run as the hive system user. The default scheduler queue mapping is one queue per user. This is problematic for users who use the queues to regulate and track their MR resource usage.

      Yarn exposes an API to retrieve the fair scheduler queue mapping, which we can use to set the appropriate MR queue for the current user.

      Attachments

        1. HIVE-8424.1.patch
          15 kB
          Mohit Sabharwal
        2. HIVE-8424.2.patch
          14 kB
          Mohit Sabharwal
        3. HIVE-8424.3.patch
          14 kB
          Mohit Sabharwal
        4. HIVE-8424.patch
          15 kB
          Mohit Sabharwal

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mohitsabharwal Mohit Sabharwal Assign to me
            mohitsabharwal Mohit Sabharwal
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment