Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3581

Prevent memory intensive user tasks from taking down nodes

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.19.0
    • None
    • None
    • Reviewed
    • Hide
      Added the ability to kill process trees transgressing memory limits. TaskTracker uses the configuration parameters introduced in HADOOP-3759. In addition, mapred.tasktracker.taskmemorymanager.monitoring-interval specifies the interval for which TT waits between cycles of monitoring tasks' memory usage, and mapred.tasktracker.procfsbasedprocesstree.sleeptime-before-sigkill specifies the time TT waits for sending a SIGKILL to a process-tree that has overrun memory limits, after it has been sent a SIGTERM.
      Show
      Added the ability to kill process trees transgressing memory limits. TaskTracker uses the configuration parameters introduced in HADOOP-3759 . In addition, mapred.tasktracker.taskmemorymanager.monitoring-interval specifies the interval for which TT waits between cycles of monitoring tasks' memory usage, and mapred.tasktracker.procfsbasedprocesstree.sleeptime-before-sigkill specifies the time TT waits for sending a SIGKILL to a process-tree that has overrun memory limits, after it has been sent a SIGTERM.

    Description

      Sometimes user Map/Reduce applications can get extremely memory intensive, maybe due to some inadvertent bugs in the user code, or the amount of data processed. When this happens, the user tasks start to interfere with the proper execution of other processes on the node, including other Hadoop daemons like the DataNode and TaskTracker. Thus, the node would become unusable for any Hadoop tasks. There should be a way to prevent such tasks from bringing down the node.

      Attachments

        1. patch_3581_0.1.txt
          14 kB
          Vinod Kumar Vavilapalli
        2. patch_3581_3.3.txt
          26 kB
          Vinod Kumar Vavilapalli
        3. patch_3581_4.3.txt
          32 kB
          Vinod Kumar Vavilapalli
        4. patch_3581_4.4.txt
          32 kB
          Vinod Kumar Vavilapalli
        5. patch_3581_5.0.txt
          40 kB
          Vinod Kumar Vavilapalli
        6. patch_3581_5.2.txt
          44 kB
          Vinod Kumar Vavilapalli
        7. HADOOP-3581.6.0.txt
          51 kB
          Vinod Kumar Vavilapalli
        8. HADOOP-3581-final.txt
          51 kB
          Vinod Kumar Vavilapalli
        9. HADOOP-3581.20080901.2.txt
          46 kB
          Vinod Kumar Vavilapalli
        10. HADOOP-3581.20080902.txt
          46 kB
          Vinod Kumar Vavilapalli
        11. HADOOP-3581.20080904.txt
          48 kB
          Vinod Kumar Vavilapalli
        12. HADOOP-3581.20080905.txt
          48 kB
          Vinod Kumar Vavilapalli
        13. HADOOP-3581.20080908.txt
          46 kB
          Vinod Kumar Vavilapalli

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-3759 Provide ability to run memory intensive jobs without affecting other running tasks on the nodes

          • Major - Major loss of function.
          • Closed
          is part of

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-3444 Implementing a Resource Manager (V1) for Hadoop

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              vinodkv Vinod Kumar Vavilapalli
              yhemanth Hemanth Yamijala
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: