Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9640 RPC Congestion Control with FairCallQueue
  3. HADOOP-10280

Make Schedulables return a configurable identity of user or group

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.4.0
    • None
    • None
    • Reviewed

    Description

      In order to intelligently schedule incoming calls, we need to know what identity it falls under.

      We do this by defining the Schedulable interface, which has one method, getIdentity(IdentityType idType)

      The scheduler can then query a Schedulable object for its identity, depending on what idType is.

      For example:
      Call 1: Made by user=Alice, group=admins
      Call 2: Made by user=Bob, group=admins
      Call 3: Made by user=Carlos, group=users
      Call 4: Made by user=Alice, group=admins

      Depending on what the identity is, we would treat these requests differently. If we query on Username, we can bucket these 4 requests into 3 sets for Alice, Bob, and Carlos. If we query on Groupname, we can bucket these 4 requests into 2 sets for admins and users.

      In this initial version, idType can be username or primary group. In future versions, it could be jobID, request class (read or write), or some explicit QoS field. These are user-defined, and will be reloaded on callqueue refresh.

      Attachments

        1. HADOOP-10280.patch
          4 kB
          Chris Li
        2. HADOOP-10280.patch
          7 kB
          Chris Li
        3. HADOOP-10280.patch
          11 kB
          Chris Li
        4. HADOOP-10280.patch
          10 kB
          Chris Li

        Activity

          People

            chrilisf Chris Li
            chrilisf Chris Li
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: