Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3581

Prevent memory intensive user tasks from taking down nodes

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Added the ability to kill process trees transgressing memory limits. TaskTracker uses the configuration parameters introduced in HADOOP-3759. In addition, mapred.tasktracker.taskmemorymanager.monitoring-interval specifies the interval for which TT waits between cycles of monitoring tasks' memory usage, and mapred.tasktracker.procfsbasedprocesstree.sleeptime-before-sigkill specifies the time TT waits for sending a SIGKILL to a process-tree that has overrun memory limits, after it has been sent a SIGTERM.
      Show
      Added the ability to kill process trees transgressing memory limits. TaskTracker uses the configuration parameters introduced in HADOOP-3759 . In addition, mapred.tasktracker.taskmemorymanager.monitoring-interval specifies the interval for which TT waits between cycles of monitoring tasks' memory usage, and mapred.tasktracker.procfsbasedprocesstree.sleeptime-before-sigkill specifies the time TT waits for sending a SIGKILL to a process-tree that has overrun memory limits, after it has been sent a SIGTERM.

      Description

      Sometimes user Map/Reduce applications can get extremely memory intensive, maybe due to some inadvertent bugs in the user code, or the amount of data processed. When this happens, the user tasks start to interfere with the proper execution of other processes on the node, including other Hadoop daemons like the DataNode and TaskTracker. Thus, the node would become unusable for any Hadoop tasks. There should be a way to prevent such tasks from bringing down the node.

        Attachments

        1. patch_3581_5.2.txt
          44 kB
          Vinod Kumar Vavilapalli
        2. patch_3581_5.0.txt
          40 kB
          Vinod Kumar Vavilapalli
        3. patch_3581_4.4.txt
          32 kB
          Vinod Kumar Vavilapalli
        4. patch_3581_4.3.txt
          32 kB
          Vinod Kumar Vavilapalli
        5. patch_3581_3.3.txt
          26 kB
          Vinod Kumar Vavilapalli
        6. patch_3581_0.1.txt
          14 kB
          Vinod Kumar Vavilapalli
        7. HADOOP-3581-final.txt
          51 kB
          Vinod Kumar Vavilapalli
        8. HADOOP-3581.6.0.txt
          51 kB
          Vinod Kumar Vavilapalli
        9. HADOOP-3581.20080908.txt
          46 kB
          Vinod Kumar Vavilapalli
        10. HADOOP-3581.20080905.txt
          48 kB
          Vinod Kumar Vavilapalli
        11. HADOOP-3581.20080904.txt
          48 kB
          Vinod Kumar Vavilapalli
        12. HADOOP-3581.20080902.txt
          46 kB
          Vinod Kumar Vavilapalli
        13. HADOOP-3581.20080901.2.txt
          46 kB
          Vinod Kumar Vavilapalli

          Issue Links

            Activity

              People

              • Assignee:
                vinodkv Vinod Kumar Vavilapalli
                Reporter:
                yhemanth Hemanth Yamijala
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: