Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-893

Provide an ability to refresh queue configuration without restart.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Extended the framework's refresh-queue mechanism to support refresh of scheduler specific queue properties and implemented this refresh operation for some of the capacity scheduler properties. With this feature, one can refresh some of the capacity-scheduler's queue related properties - queue capacities, user-limits per queue, max map/reduce capacity and max-jobs per user to initialize while the system is running and without restarting JT. Even after this, some features like changing enable/disable priorities, adding/removing queues are not supported in capacity-scheduler.
      Show
      Extended the framework's refresh-queue mechanism to support refresh of scheduler specific queue properties and implemented this refresh operation for some of the capacity scheduler properties. With this feature, one can refresh some of the capacity-scheduler's queue related properties - queue capacities, user-limits per queue, max map/reduce capacity and max-jobs per user to initialize while the system is running and without restarting JT. Even after this, some features like changing enable/disable priorities, adding/removing queues are not supported in capacity-scheduler.

      Description

      While administering a cluster using multiple queues, administrators feel a need to refresh queue properties on the fly without needing to restart the JobTracker. This is partially supported for some properties such as queue ACLs (HADOOP-5396) and state (HADOOP-5913). The idea is to extend the facility to refresh other queue properties as well, including scheduler properties.

      1. MAPREDUCE-893-20090915.1.txt
        128 kB
        Vinod Kumar Vavilapalli
      2. MAPREDUCE-893-20090917.2.txt
        172 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-893-20090917.4.txt
        170 kB
        Vinod Kumar Vavilapalli
      4. MAPREDUCE-893-20090918.2.txt
        171 kB
        Vinod Kumar Vavilapalli
      5. MAPREDUCE-893-20090918.over.849.patch
        168 kB
        Vinod Kumar Vavilapalli
      6. MAPREDUCE-893-20090918.txt
        171 kB
        Vinod Kumar Vavilapalli
      7. MAPREDUCE-893-7.patch
        159 kB
        rahul k singh

        Issue Links

          Activity

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Hemanth Yamijala
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development