Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-707

Provide a jobconf property for explicitly assigning a job to a pool

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: contrib/fair-share
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      add mapred.fairscheduler.pool property to define which pool a job belongs to.

      Description

      A common use case of the fair scheduler is to have one pool per user, but then to define some special pools for various production jobs, import jobs, etc. Therefore, it would be nice if jobs went by default to the pool of the user who submitted them, but there was a setting to explicitly place a job in another pool. Today, this can be achieved through a sort of trick in the JobConf:

      <property>
        <name>mapred.fairscheduler.poolnameproperty</name>
        <value>pool.name</value>
      </property>
      
      <property>
        <name>pool.name</name>
        <value>${user.name}</value>
      </property>
      

      This JIRA proposes to add a property called mapred.fairscheduler.pool that allows a job to be placed directly into a pool, avoiding the need for this trick.

      1. MAPREDUCE-707-2-apache.patch
        8 kB
        Alan Heirich
      2. MAPREDUCE-707-1-apache.patch
        8 kB
        Alan Heirich
      3. MAPREDUCE-707-apache.patch
        6 kB
        Alan Heirich

        Activity

          People

          • Assignee:
            Alan Heirich
            Reporter:
            Matei Zaharia
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development