Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-144

TaskMemoryManager should log process-tree's status while killing tasks.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE".

      This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.
      Show
      Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE". This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.

      Description

      This helps a lot in debugging why a particular task has gone beyond memory limits.

      1. MAPREDUCE-144-20090907.internal.txt
        13 kB
        Hemanth Yamijala
      2. MAPREDUCE-144-20090908.txt
        13 kB
        Vinod Kumar Vavilapalli
      3. MAPREDUCE-144-20090903.1.txt
        13 kB
        Vinod Kumar Vavilapalli
      4. MAPREDUCE-144-20090903.txt
        72 kB
        Vinod Kumar Vavilapalli
      5. MAPREDUCE-144-20090902.txt
        7 kB
        Vinod Kumar Vavilapalli
      6. HADOOP-5570-1.patch
        6 kB
        Sreekanth Ramakrishnan
      7. HADOOP-5570-20090622.txt
        6 kB
        Vinod Kumar Vavilapalli
      8. HADOOP-5570-20090619.txt
        6 kB
        Vinod Kumar Vavilapalli
      9. HADOOP-5570-20090615.txt
        5 kB
        Vinod Kumar Vavilapalli
      10. HADOOP-5570-20090614.txt
        5 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Vinod Kumar Vavilapalli created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Assignee Vinod K V [ vinodkv ]
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090614.txt [ 12408123 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090615.txt [ 12408234 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090619.txt [ 12408459 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090622.txt [ 12408803 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Sreekanth Ramakrishnan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Sreekanth Ramakrishnan made changes -
          Attachment HADOOP-5570-1.patch [ 12410773 ]
          Sreekanth Ramakrishnan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Owen O'Malley made changes -
          Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
          Key HADOOP-5570 MAPREDUCE-144
          Component/s mapred [ 12310690 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is blocked by HADOOP-6156 [ HADOOP-6156 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is blocked by HADOOP-6156 [ HADOOP-6156 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is blocked by HADOOP-6230 [ HADOOP-6230 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090902.txt [ 12418347 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090903.txt [ 12418511 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090903.1.txt [ 12418513 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090908.txt [ 12418894 ]
          Hemanth Yamijala made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hemanth Yamijala made changes -
          Fix Version/s 0.21.0 [ 12314045 ]
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Hemanth Yamijala made changes -
          Attachment MAPREDUCE-144-20090907.internal.txt [ 12418917 ]
          Vinod Kumar Vavilapalli made changes -
          Component/s tasktracker [ 12312906 ]
          Vinod Kumar Vavilapalli made changes -
          Release Note Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE".

          This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development