Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-853 Support a hierarchy of queues in the Map/Reduce framework
  3. MAPREDUCE-861

Modify queue configuration format and parsing to support a hierarchy of queues.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • jobtracker
    • None
    • Reviewed
    • Hide
      Added support for hierarchical queues in the Map/Reduce framework with the following changes:
      - mapred-queues.xml is modified to a new XML template as mentioned in the JIRA.
      - Modified JobQueueInfo to contain a handle to child queues.
      - Added new APIs in the client to get 'root' queues, so that the entire hierarchy of queues can be iterated.
      -Added new APIs to get the child queues for a given queue .
      Show
      Added support for hierarchical queues in the Map/Reduce framework with the following changes: - mapred-queues.xml is modified to a new XML template as mentioned in the JIRA. - Modified JobQueueInfo to contain a handle to child queues. - Added new APIs in the client to get 'root' queues, so that the entire hierarchy of queues can be iterated. -Added new APIs to get the child queues for a given queue .

    Description

      MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce framework. This JIRA is for defining changes to the configuration related to queues.

      The current format for defining a queue and its properties is as follows: mapred.queue.<queue-name>.<property-name>. For e.g. mapred.queue.<queue-name>.acl-submit-job. The reason for using this verbose format was to be able to reuse the Configuration parser in Hadoop. However, administrators currently using the queue configuration have already indicated a very strong desire for a more manageable format. Since, this becomes more unwieldy with hierarchical queues, the time may be good to introduce a new format for representing queue configuration.

      Attachments

        1. MAPREDUCE-861-1.patch
          114 kB
          rahul k singh
        2. MAPREDUCE-861-2.patch
          127 kB
          Sreekanth Ramakrishnan
        3. MAPREDUCE-861-3.patch
          172 kB
          rahul k singh
        4. MAPREDUCE-861-4.patch
          168 kB
          rahul k singh
        5. MAPREDUCE-861-5.patch
          166 kB
          rahul k singh
        6. MAPREDUCE-861-6.patch
          178 kB
          rahul k singh
        7. MAPREDUCE-861-7.patch
          179 kB
          rahul k singh

        Issue Links

          Activity

            People

              rksingh rahul k singh
              yhemanth Hemanth Yamijala
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: