Nathan Roberts added a comment - 05/Apr/16 15:08 Simple fix that falls back to sessionID if process has become owned by init. Seemed safest low risk change. Other options might be: Only use sessionID to build process tree Use container cgroup (cgroup.procs) if available/configured.

Nathan Roberts added a comment - 05/Apr/16 15:11 As a note, we were seeing this with slider applications. I didn't investigate far enough to know if all slider applications escape or if this was just a a characteristic of this particular application.

Hadoop QA added a comment - 05/Apr/16 15:39 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 38s trunk passed +1 compile 0m 22s trunk passed with JDK v1.8.0_77 +1 compile 0m 26s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 31s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 8s trunk passed +1 javadoc 0m 26s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 34s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 26s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_77 +1 javac 0m 21s the patch passed +1 compile 0m 24s the patch passed with JDK v1.7.0_95 +1 javac 0m 24s the patch passed -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common: patch generated 6 new + 35 unchanged - 0 fixed = 41 total (was 35) +1 mvnsite 0m 28s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 18s the patch passed +1 javadoc 0m 25s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 32s the patch passed with JDK v1.7.0_95 +1 unit 2m 1s hadoop-yarn-common in the patch passed with JDK v1.8.0_77. +1 unit 2m 17s hadoop-yarn-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 20m 39s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12797089/YARN-4834.001.patch JIRA Issue YARN-4834 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bcba00a3970f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9174645 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10966/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10966/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common Console output https://builds.apache.org/job/PreCommit-YARN-Build/10966/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.

Jason Lowe added a comment - 02/May/16 17:46 +1 to using the session ID to track the processes for a container. Ideally if we're using cgroups we should use that instead, but in the interim this would be a significant improvement. I'm not thrilled with keeping the double-pass logic leftover from the parent-child tree calculations. The session ID approach would only require a single pass. However that's a significant rewrite of the code, and I can appreciate keeping the changes to a minimum to fix this bug. We can file a followup JIRA to simplify the logic. +1 lgtm. Will commit this tomorrow if there are no objections.

Hudson added a comment - 03/May/16 17:41 FAILURE: Integrated in Hadoop-trunk-Commit #9707 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9707/ ) YARN-4834 . ProcfsBasedProcessTree doesn't track daemonized processes. (jlowe: rev c6b48391680c1b81a86aabc3ad4c725bfade6d2e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java

Jason Lowe added a comment - 03/May/16 17:42 Thanks, Nathan! I committed this to trunk, branch-2, branch-2.8, and branch-2.7.

Vinod Kumar Vavilapalli added a comment - 25/Aug/16 23:19 Closing the JIRA as part of 2.7.3 release.

Grant Sohn added a comment - 27/Oct/16 20:30 Regarding the 'kill – ...': Ubuntu kill doesn't support "--" but recommends killall instead.