Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-144

TaskMemoryManager should log process-tree's status while killing tasks.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE".

      This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.
      Show
      Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE". This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.

      Description

      This helps a lot in debugging why a particular task has gone beyond memory limits.

      1. MAPREDUCE-144-20090908.txt
        13 kB
        Vinod Kumar Vavilapalli
      2. MAPREDUCE-144-20090907.internal.txt
        13 kB
        Hemanth Yamijala
      3. MAPREDUCE-144-20090903.txt
        72 kB
        Vinod Kumar Vavilapalli
      4. MAPREDUCE-144-20090903.1.txt
        13 kB
        Vinod Kumar Vavilapalli
      5. MAPREDUCE-144-20090902.txt
        7 kB
        Vinod Kumar Vavilapalli
      6. HADOOP-5570-20090622.txt
        6 kB
        Vinod Kumar Vavilapalli
      7. HADOOP-5570-20090619.txt
        6 kB
        Vinod Kumar Vavilapalli
      8. HADOOP-5570-20090615.txt
        5 kB
        Vinod Kumar Vavilapalli
      9. HADOOP-5570-20090614.txt
        5 kB
        Vinod Kumar Vavilapalli
      10. HADOOP-5570-1.patch
        6 kB
        Sreekanth Ramakrishnan

        Issue Links

          Activity

          Vinod Kumar Vavilapalli created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Assignee Vinod K V [ vinodkv ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Attaching a patch to fix the issue. This patch reads /proc/pid/cmdline to get the command line of each process and uses this to get the process-tree's dump. The dump is logged both on the TaskTracker and is displayed on the web-ui in the Errors tab. Added a simple test-case and also tested this on a local cluster.

          Show
          Vinod Kumar Vavilapalli added a comment - Attaching a patch to fix the issue. This patch reads /proc/pid/cmdline to get the command line of each process and uses this to get the process-tree's dump. The dump is logged both on the TaskTracker and is displayed on the web-ui in the Errors tab. Added a simple test-case and also tested this on a local cluster.
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090614.txt [ 12408123 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Sreekanth Ramakrishnan added a comment -

          Changes look fine to me. +1 to patch

          Show
          Sreekanth Ramakrishnan added a comment - Changes look fine to me. +1 to patch
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Small changes to the patch. The process-tree dump propagates back to the JobClient. So, pretty-printing the process-tree dump now to reduce clutter on the client. Also adding tip-id in the process-tree dump for quick reference.

          Show
          Vinod Kumar Vavilapalli added a comment - Small changes to the patch. The process-tree dump propagates back to the JobClient. So, pretty-printing the process-tree dump now to reduce clutter on the client. Also adding tip-id in the process-tree dump for quick reference.
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090615.txt [ 12408234 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12408234/HADOOP-5570-20090615.txt
          against trunk revision 775492.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12408234/HADOOP-5570-20090615.txt against trunk revision 775492. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/console This message is automatically generated.
          Hide
          Sreekanth Ramakrishnan added a comment -

          +1 to the latest patch.

          Show
          Sreekanth Ramakrishnan added a comment - +1 to the latest patch.
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Another small change to the patch. This adds to the dump the memory usage of each process in the process-tree at the time of killing.

          Show
          Vinod Kumar Vavilapalli added a comment - Another small change to the patch. This adds to the dump the memory usage of each process in the process-tree at the time of killing.
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090619.txt [ 12408459 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Patch fixing the test case.

          Show
          Vinod Kumar Vavilapalli added a comment - Patch fixing the test case.
          Vinod Kumar Vavilapalli made changes -
          Attachment HADOOP-5570-20090622.txt [ 12408803 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12408803/HADOOP-5570-20090622.txt
          against trunk revision 778388.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12408803/HADOOP-5570-20090622.txt against trunk revision 778388. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/console This message is automatically generated.
          Sreekanth Ramakrishnan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Sreekanth Ramakrishnan added a comment -

          Attaching patch merging with lastest trunk.

          Show
          Sreekanth Ramakrishnan added a comment - Attaching patch merging with lastest trunk.
          Sreekanth Ramakrishnan made changes -
          Attachment HADOOP-5570-1.patch [ 12410773 ]
          Sreekanth Ramakrishnan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Owen O'Malley made changes -
          Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
          Key HADOOP-5570 MAPREDUCE-144
          Component/s mapred [ 12310690 ]
          Hide
          Chris Douglas added a comment -

          The patch needs to be regenerated following the project split.

          Show
          Chris Douglas added a comment - The patch needs to be regenerated following the project split.
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is blocked by HADOOP-6156 [ HADOOP-6156 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          This patch touches ProcfsBasedProcessTree which is intended to be moved into mapreduce projects via HADOOP-6156. Linking the issues.

          Show
          Vinod Kumar Vavilapalli added a comment - This patch touches ProcfsBasedProcessTree which is intended to be moved into mapreduce projects via HADOOP-6156 . Linking the issues.
          Vinod Kumar Vavilapalli made changes -
          Link This issue is blocked by HADOOP-6156 [ HADOOP-6156 ]
          Vinod Kumar Vavilapalli made changes -
          Link This issue is blocked by HADOOP-6230 [ HADOOP-6230 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Attaching an updated patch to reflect the project split. Also added parent pid to the dump.

          The tree dump would look something like below"

          TaskTree [pid=20096,tipID=attempt_20090902122532859_0001_m_000004_0] is running beyond memory-
          limits. Current usage : 426889216bytes. Limit : 1048576bytes. Killing task. 
          Dump of the process-tree for attempt_20090902122532859_0001_m_000004_0 : 
                  |- $pid $ppid $cmd_name $vmem_usage $cmd_line
                  |- 20130 20128 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data
          /testProcessTree_shellScript_-2044420742.sh 0 
                  |- 20128 20126 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data
          /testProcessTree_shellScript_-2044420742.sh 1 
                  |- 20116 20096 (bash) 4542464bytes bash -c  echo $$ > /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build
          /test/data/testProcessTree_pidFile_-1719677031.pid; sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data
          /testProcessTree_shellScript_-2044420742.sh 6; 
                  |- 20117 20116 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data
          /testProcessTree_shellScript_-2044420742.sh 6 
                  |- 20132 20130 (sleep) 3108864bytes sleep 5 
                  |- 20120 20117 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data
          /testProcessTree_shellScript_-2044420742.sh 5 
                  ....
          
          Show
          Vinod Kumar Vavilapalli added a comment - Attaching an updated patch to reflect the project split. Also added parent pid to the dump. The tree dump would look something like below" TaskTree [pid=20096,tipID=attempt_20090902122532859_0001_m_000004_0] is running beyond memory- limits. Current usage : 426889216bytes. Limit : 1048576bytes. Killing task. Dump of the process-tree for attempt_20090902122532859_0001_m_000004_0 : |- $pid $ppid $cmd_name $vmem_usage $cmd_line |- 20130 20128 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data /testProcessTree_shellScript_-2044420742.sh 0 |- 20128 20126 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data /testProcessTree_shellScript_-2044420742.sh 1 |- 20116 20096 (bash) 4542464bytes bash -c echo $$ > /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build /test/data/testProcessTree_pidFile_-1719677031.pid; sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data /testProcessTree_shellScript_-2044420742.sh 6; |- 20117 20116 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data /testProcessTree_shellScript_-2044420742.sh 6 |- 20132 20130 (sleep) 3108864bytes sleep 5 |- 20120 20117 (sh) 1916928bytes sh /home/vinodkv/Workspace/eclipse-workspace/hadoop-mapreduce/build/test/data /testProcessTree_shellScript_-2044420742.sh 5 ....
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090902.txt [ 12418347 ]
          Hide
          Hemanth Yamijala added a comment -

          Some comments:

          • Firstly, it appears cmdline could be empty if the whole process is swapped out. I suppose this is possible. Is the method we use reliable then ?
          • I think getCmdLine can return "Unknown" rather than empty when it can't find a command line.
          • Also, the replacement of '\0' with space should happen in getCmdLine, that too, only when we read the line, rather than getProcessTreeDump(), because it is more relevant only there.
          • I think it will be useful to also show session id and process group id, so that we can catch for mysterious changes to these if tasks are doing them.
          • Regarding the test case, it seems we can get away with a unit test for this, just like testVirtualMemoryForProcessTree(), which would be more controlled and reliable rather than on the relying on system behavior of procfs. That way, even if there is a problem, we can differentiate between system incompatibilities and bugs in code. I am fine with the existing test, but would like to treat that as a smoke test of the feature, but have a unit test for fast runs.
          Show
          Hemanth Yamijala added a comment - Some comments: Firstly, it appears cmdline could be empty if the whole process is swapped out. I suppose this is possible. Is the method we use reliable then ? I think getCmdLine can return "Unknown" rather than empty when it can't find a command line. Also, the replacement of '\0' with space should happen in getCmdLine, that too, only when we read the line, rather than getProcessTreeDump(), because it is more relevant only there. I think it will be useful to also show session id and process group id, so that we can catch for mysterious changes to these if tasks are doing them. Regarding the test case, it seems we can get away with a unit test for this, just like testVirtualMemoryForProcessTree(), which would be more controlled and reliable rather than on the relying on system behavior of procfs. That way, even if there is a problem, we can differentiate between system incompatibilities and bugs in code. I am fine with the existing test, but would like to treat that as a smoke test of the feature, but have a unit test for fast runs.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Attaching patch.

          Firstly, it appears cmdline could be empty if the whole process is swapped out. I suppose this is possible. Is the method we use reliable then?

          We don't have any other simple way of determining the complete command line. Presumably, we can read it from the process's image on the disk, but the effort is not worth it. Printing N/A when we find the cmdline empty.

          I think getCmdLine can return "Unknown" rather than empty when it can't find a command line.

          Printing N/A here too.

          Incorporated the rest of the comments too.

          In sum, the dump looks like

                  |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE
                  |- 100 1 100 100 (proc1) 100000 proc1 arg1 arg2
                  |- 200 100 100 100 (proc2) 200000 proc2 arg3 arg4
                  |- 500 400 100 100 (proc5) 400000 proc5 arg9 arg10
                  |- 400 200 100 100 (proc4) 400000 proc4 arg7 arg8
                  |- 300 200 100 100 (proc3) 300000 proc3 arg5 arg6
          
          Show
          Vinod Kumar Vavilapalli added a comment - Attaching patch. Firstly, it appears cmdline could be empty if the whole process is swapped out. I suppose this is possible. Is the method we use reliable then? We don't have any other simple way of determining the complete command line. Presumably, we can read it from the process's image on the disk, but the effort is not worth it. Printing N/A when we find the cmdline empty. I think getCmdLine can return "Unknown" rather than empty when it can't find a command line. Printing N/A here too. Incorporated the rest of the comments too. In sum, the dump looks like |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE |- 100 1 100 100 (proc1) 100000 proc1 arg1 arg2 |- 200 100 100 100 (proc2) 200000 proc2 arg3 arg4 |- 500 400 100 100 (proc5) 400000 proc5 arg9 arg10 |- 400 200 100 100 (proc4) 400000 proc4 arg7 arg8 |- 300 200 100 100 (proc3) 300000 proc3 arg5 arg6
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090903.txt [ 12418511 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          The previous patch had a few stale changes. Please disregard that. Attaching a corrected patch.

          Show
          Vinod Kumar Vavilapalli added a comment - The previous patch had a few stale changes. Please disregard that. Attaching a corrected patch.
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090903.1.txt [ 12418513 ]
          Hide
          Hemanth Yamijala added a comment -

          Changes look good to me. +1.

          Show
          Hemanth Yamijala added a comment - Changes look good to me. +1.
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12418513/MAPREDUCE-144-20090903.1.txt
          against trunk revision 812287.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 1 new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418513/MAPREDUCE-144-20090903.1.txt against trunk revision 812287. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/14/console This message is automatically generated.
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Patch fixing find-bugs warning.

          Show
          Vinod Kumar Vavilapalli added a comment - Patch fixing find-bugs warning.
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-144-20090908.txt [ 12418894 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          The mapred, contrib tests passed with this patch and with a jar that avoids problems causing HADOOP-6243.

          Show
          Vinod Kumar Vavilapalli added a comment - The mapred, contrib tests passed with this patch and with a jar that avoids problems causing HADOOP-6243 .
          Hide
          Vinod Kumar Vavilapalli added a comment -

          `ant test-patch` also passed.

          Show
          Vinod Kumar Vavilapalli added a comment - `ant test-patch` also passed.
          Hide
          Hemanth Yamijala added a comment -

          Running through Hudson to try if things work, now that HADOOP-6243 changes are committed.

          Show
          Hemanth Yamijala added a comment - Running through Hudson to try if things work, now that HADOOP-6243 changes are committed.
          Hemanth Yamijala made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12418894/MAPREDUCE-144-20090908.txt
          against trunk revision 812473.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418894/MAPREDUCE-144-20090908.txt against trunk revision 812473. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/45/console This message is automatically generated.
          Hide
          Hemanth Yamijala added a comment -

          I just committed this to trunk. Thanks, Vinod !

          Show
          Hemanth Yamijala added a comment - I just committed this to trunk. Thanks, Vinod !
          Hemanth Yamijala made changes -
          Fix Version/s 0.21.0 [ 12314045 ]
          Resolution Fixed [ 1 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Hide
          Hemanth Yamijala added a comment -

          This is a patch for the Yahoo! distribution, not for committing.

          Show
          Hemanth Yamijala added a comment - This is a patch for the Yahoo! distribution, not for committing.
          Hemanth Yamijala made changes -
          Attachment MAPREDUCE-144-20090907.internal.txt [ 12418917 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #24 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/24/)
          . Includes dump of the process tree in task diagnostics when a task is killed due to exceeding memory limits. Contributed by Vinod Kumar Vavilapalli.

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #24 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/24/ ) . Includes dump of the process tree in task diagnostics when a task is killed due to exceeding memory limits. Contributed by Vinod Kumar Vavilapalli.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE".

          This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.

          Show
          Vinod Kumar Vavilapalli added a comment - Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE". This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.
          Vinod Kumar Vavilapalli made changes -
          Component/s tasktracker [ 12312906 ]
          Vinod Kumar Vavilapalli made changes -
          Release Note Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE".

          This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager.
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development