Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-516

Fix the 'cluster drain' problem in the Capacity Scheduler wrt High RAM Jobs

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      When a HighRAMJob turns up at the head of the queue, the current implementation of support for HighRAMJobs in the Capacity Scheduler has problem in that the scheduler stops assigning tasks to all TaskTrackers in the cluster until a HighRAMJob finds a suitable TaskTrackers for all its tasks.

      This causes a severe utilization problem since effectively no new tasks are allowed to run until the HighRAMJob (at the head of the queue) gets slots.

      1. mr516-ydist.patch
        170 kB
        Sreekanth Ramakrishnan
      2. HADOOP-5964_2_20090629_yhadoop.patch
        170 kB
        Arun C Murthy
      3. HADOOP-5964_2_20090629_yhadoop.patch
        170 kB
        Arun C Murthy
      4. MAPREDUCE-516-Y20.patch
        170 kB
        Sreekanth Ramakrishnan
      5. HADOOP-5964_1_20090623_yhadoop.patch
        170 kB
        Arun C Murthy
      6. HADOOP-5964-12.patch
        168 kB
        Hemanth Yamijala
      7. HADOOP-5964_11_20090623.patch
        174 kB
        Arun C Murthy
      8. HADOOP-5964-11.patch
        165 kB
        Hemanth Yamijala
      9. HADOOP-5964_10_20090622.patch
        162 kB
        Arun C Murthy
      10. HADOOP-5964_9_20090619.patch
        163 kB
        Arun C Murthy
      11. HADOOP-5964_9_20090619.patch
        164 kB
        Arun C Murthy
      12. HADOOP-5964_8_20090618.patch
        163 kB
        Arun C Murthy
      13. HADOOP-5964_7_20090618.patch
        148 kB
        Arun C Murthy
      14. HADOOP-5964_6_20090617.patch
        145 kB
        Arun C Murthy
      15. HADOOP-5964_4_20090615.patch
        84 kB
        Arun C Murthy
      16. HADOOP-5964_2_20090609.patch
        68 kB
        Arun C Murthy
      17. HADOOP-5964_1_20090608.patch
        54 kB
        Arun C Murthy
      18. HADOOP-5964_0_20090602.patch
        31 kB
        Arun C Murthy

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Arun C Murthy
              Reporter:
              Arun C Murthy
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development