Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4523

Enhance how memory-intensive user tasks are handled

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-3581 monitors each Hadoop task to see if its memory usage (which includes usage of any tasks spawned by it and so on) is within a per-task limit. If the task's memory usage goes over its limit, the task is killed. This, by itself, is not enough to prevent badly behaving jobs from bringing down nodes. What is also needed is the ability to make sure that the sum total of VM usage of all Hadoop tasks does not exceed a certain limit.

        Attachments

        1. HADOOP-4523-200811-05.txt
          12 kB
          Vinod Kumar Vavilapalli
        2. HADOOP-4523-200811-06.txt
          17 kB
          Vinod Kumar Vavilapalli
        3. HADOOP-4523-20081110.txt
          19 kB
          Vinod Kumar Vavilapalli
        4. HADOOP-4523-20081113.txt
          22 kB
          Vinod Kumar Vavilapalli
        5. HADOOP-4523-20081118.txt
          22 kB
          Vinod Kumar Vavilapalli

          Issue Links

            Activity

              People

              • Assignee:
                vinodkv Vinod Kumar Vavilapalli
                Reporter:
                vivekr Vivek Ratan
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: