Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5121

fix some container-executor portability issues

    Details

      Description

      container-executor has some issues that are preventing it from even compiling on the OS X jenkins instance. Let's fix those. While we're there, let's also try to take care of some of the other portability problems that have crept in over the years, since it used to work great on Solaris but now doesn't.

      1. YARN-6698-branch-2.7-01.patch
        52 kB
        Akira Ajisaka
      2. YARN-5121.08.patch
        54 kB
        Allen Wittenauer
      3. YARN-5121.07.patch
        54 kB
        Allen Wittenauer
      4. YARN-5121.06.patch
        53 kB
        Allen Wittenauer
      5. YARN-5121.04.patch
        53 kB
        Allen Wittenauer
      6. YARN-5121.03.patch
        44 kB
        Allen Wittenauer
      7. YARN-5121.02.patch
        46 kB
        Allen Wittenauer
      8. YARN-5121.01.patch
        47 kB
        Allen Wittenauer
      9. YARN-5121.00.patch
        38 kB
        Allen Wittenauer

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment - - edited

          -00:

          • add some missing routines for OS X 10.9
          • replace get_executable with something portable
          • lock out cgroup support on non-Linux platforms
          • add necessary license and RAT exclusion handling
          • change the optional defines so they are actually legal for clang
          • fix a ton of whitespace issues
          Show
          aw Allen Wittenauer added a comment - - edited -00: add some missing routines for OS X 10.9 replace get_executable with something portable lock out cgroup support on non-Linux platforms add necessary license and RAT exclusion handling change the optional defines so they are actually legal for clang fix a ton of whitespace issues
          Hide
          aw Allen Wittenauer added a comment -

          Ping Chris Nauroth, since he and I were talking about this problem.

          Show
          aw Allen Wittenauer added a comment - Ping Chris Nauroth , since he and I were talking about this problem.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          +1 @author 0m 00s The patch does not contain any @author tags.
          +1 test4tests 0m 00s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 26s Maven dependency ordering for branch
          +1 mvninstall 7m 47s trunk passed
          -1 compile 3m 49s root in trunk failed.
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 7m 48s the patch passed
          +1 compile 7m 46s the patch passed
          -1 cc 7m 46s root generated 11 new + 17 unchanged - 9 fixed = 28 total (was 26)
          -1 javac 7m 46s root generated 526 new + 172 unchanged - 0 fixed = 698 total (was 172)
          -1 unit 15m 24s root in the patch failed.
          43m 55s



          Reason Tests
          Failed junit tests hadoop.fs.TestSymlinkLocalFSFileContext
            hadoop.fs.TestSymlinkLocalFSFileSystem
            hadoop.net.unix.TestDomainSocket



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805475/YARN-5121.00.patch
          JIRA Issue YARN-5121
          Optional Tests compile javac mvninstall unit cc
          uname Darwin Gavins-Mac-mini.local 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64
          Build tool maven
          Personality /Users/jenkins/jenkins-home/workspace/Precommit-HADOOP-OSX/patchprocess/apache-yetus-21ed107/precommit/personality/hadoop.sh
          git revision trunk / d8c1fd1
          Default Java 1.8.0_74
          compile https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/branch-compile-root.txt
          cc https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/diff-compile-cc-root.txt
          javac https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/diff-compile-javac-root.txt
          unit https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/Precommit-HADOOP-OSX/18/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/Precommit-HADOOP-OSX/18/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment +1 @author 0m 00s The patch does not contain any @author tags. +1 test4tests 0m 00s The patch appears to include 1 new or modified test files. 0 mvndep 0m 26s Maven dependency ordering for branch +1 mvninstall 7m 47s trunk passed -1 compile 3m 49s root in trunk failed. 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 7m 48s the patch passed +1 compile 7m 46s the patch passed -1 cc 7m 46s root generated 11 new + 17 unchanged - 9 fixed = 28 total (was 26) -1 javac 7m 46s root generated 526 new + 172 unchanged - 0 fixed = 698 total (was 172) -1 unit 15m 24s root in the patch failed. 43m 55s Reason Tests Failed junit tests hadoop.fs.TestSymlinkLocalFSFileContext   hadoop.fs.TestSymlinkLocalFSFileSystem   hadoop.net.unix.TestDomainSocket Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805475/YARN-5121.00.patch JIRA Issue YARN-5121 Optional Tests compile javac mvninstall unit cc uname Darwin Gavins-Mac-mini.local 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 Build tool maven Personality /Users/jenkins/jenkins-home/workspace/Precommit-HADOOP-OSX/patchprocess/apache-yetus-21ed107/precommit/personality/hadoop.sh git revision trunk / d8c1fd1 Default Java 1.8.0_74 compile https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/branch-compile-root.txt cc https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/diff-compile-cc-root.txt javac https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/diff-compile-javac-root.txt unit https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/Precommit-HADOOP-OSX/18/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/Precommit-HADOOP-OSX/18/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/Precommit-HADOOP-OSX/18/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          Unit test failures are known.

          The increase in errors is directly correlated with the build no longer prematurely exiting.

          Show
          aw Allen Wittenauer added a comment - Unit test failures are known. The increase in errors is directly correlated with the build no longer prematurely exiting.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 12s Maven dependency ordering for branch
          +1 mvninstall 6m 3s trunk passed
          +1 compile 6m 33s trunk passed
          +1 mvnsite 8m 30s trunk passed
          +1 mvneclipse 1m 25s trunk passed
          +1 javadoc 5m 33s trunk passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 6m 46s the patch passed
          +1 compile 6m 23s the patch passed
          +1 cc 6m 23s the patch passed
          +1 javac 6m 23s the patch passed
          +1 mvnsite 8m 8s the patch passed
          +1 mvneclipse 0m 33s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 javadoc 5m 39s the patch passed
          -1 unit 124m 14s root in the patch failed.
          +1 asflicense 0m 29s Patch does not generate ASF License warnings.
          181m 33s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805475/YARN-5121.00.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux bda5ec503747 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 500e946
          Default Java 1.8.0_91
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11609/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11609/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11609/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11609/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 6m 3s trunk passed +1 compile 6m 33s trunk passed +1 mvnsite 8m 30s trunk passed +1 mvneclipse 1m 25s trunk passed +1 javadoc 5m 33s trunk passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 6m 46s the patch passed +1 compile 6m 23s the patch passed +1 cc 6m 23s the patch passed +1 javac 6m 23s the patch passed +1 mvnsite 8m 8s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 javadoc 5m 39s the patch passed -1 unit 124m 14s root in the patch failed. +1 asflicense 0m 29s Patch does not generate ASF License warnings. 181m 33s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805475/YARN-5121.00.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux bda5ec503747 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 500e946 Default Java 1.8.0_91 unit https://builds.apache.org/job/PreCommit-YARN-Build/11609/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11609/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11609/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11609/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          I'm changing the hadoop-trunk-osx-java8 and the Precommit-HADOOP-OSX job so that it always applies this patch.

          Show
          aw Allen Wittenauer added a comment - I'm changing the hadoop-trunk-osx-java8 and the Precommit-HADOOP-OSX job so that it always applies this patch.
          Hide
          aw Allen Wittenauer added a comment - - edited

          Looks like test-container-executor has quite a few problems:

          • assumes /tmp is a fixed directory path (it's not on OS X)
          • can't run two test-container-executors on the same machine due to hard-coded usage of /tmp
          • various outputs of routines aren't checked for a (legal) null, so causes core dumps when that value is returned

          So, basically, test-container-executor is pretty unreliable. Whee.

          Show
          aw Allen Wittenauer added a comment - - edited Looks like test-container-executor has quite a few problems: assumes /tmp is a fixed directory path (it's not on OS X) can't run two test-container-executors on the same machine due to hard-coded usage of /tmp various outputs of routines aren't checked for a (legal) null, so causes core dumps when that value is returned So, basically, test-container-executor is pretty unreliable. Whee.
          Hide
          aw Allen Wittenauer added a comment -

          -01:

          • fix some of the test-container-executor issues too
          Show
          aw Allen Wittenauer added a comment - -01: fix some of the test-container-executor issues too
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          +1 @author 0m 00s The patch does not contain any @author tags.
          +1 test4tests 0m 00s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 7m 54s trunk passed
          -1 compile 4m 08s root in trunk failed.
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 8m 10s the patch passed
          +1 compile 7m 56s the patch passed
          -1 cc 7m 56s root generated 10 new + 18 unchanged - 8 fixed = 28 total (was 26)
          -1 javac 7m 56s root generated 526 new + 171 unchanged - 0 fixed = 697 total (was 171)
          -1 unit 15m 35s root in the patch failed.
          45m 00s



          Reason Tests
          Failed junit tests hadoop.fs.TestSymlinkLocalFSFileContext
            hadoop.fs.TestSymlinkLocalFSFileSystem
            hadoop.net.unix.TestDomainSocket



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806152/YARN-5121.01.patch
          JIRA Issue YARN-5121
          Optional Tests compile javac mvninstall unit cc
          uname Darwin Gavins-Mac-mini.local 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64
          Build tool maven
          Personality /Users/jenkins/jenkins-home/workspace/Precommit-HADOOP-OSX/patchprocess/apache-yetus-bde9590/precommit/personality/hadoop.sh
          git revision trunk / 9a31e5d
          Default Java 1.8.0_74
          compile https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/branch-compile-root.txt
          cc https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/diff-compile-cc-root.txt
          javac https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/diff-compile-javac-root.txt
          unit https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/Precommit-HADOOP-OSX/24/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/Precommit-HADOOP-OSX/24/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment +1 @author 0m 00s The patch does not contain any @author tags. +1 test4tests 0m 00s The patch appears to include 1 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 54s trunk passed -1 compile 4m 08s root in trunk failed. 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 8m 10s the patch passed +1 compile 7m 56s the patch passed -1 cc 7m 56s root generated 10 new + 18 unchanged - 8 fixed = 28 total (was 26) -1 javac 7m 56s root generated 526 new + 171 unchanged - 0 fixed = 697 total (was 171) -1 unit 15m 35s root in the patch failed. 45m 00s Reason Tests Failed junit tests hadoop.fs.TestSymlinkLocalFSFileContext   hadoop.fs.TestSymlinkLocalFSFileSystem   hadoop.net.unix.TestDomainSocket Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806152/YARN-5121.01.patch JIRA Issue YARN-5121 Optional Tests compile javac mvninstall unit cc uname Darwin Gavins-Mac-mini.local 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64 Build tool maven Personality /Users/jenkins/jenkins-home/workspace/Precommit-HADOOP-OSX/patchprocess/apache-yetus-bde9590/precommit/personality/hadoop.sh git revision trunk / 9a31e5d Default Java 1.8.0_74 compile https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/branch-compile-root.txt cc https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/diff-compile-cc-root.txt javac https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/diff-compile-javac-root.txt unit https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/Precommit-HADOOP-OSX/24/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/Precommit-HADOOP-OSX/24/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/Precommit-HADOOP-OSX/24/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 13s Maven dependency ordering for branch
          +1 mvninstall 6m 10s trunk passed
          +1 compile 6m 40s trunk passed
          +1 mvnsite 9m 4s trunk passed
          +1 mvneclipse 0m 41s trunk passed
          +1 javadoc 6m 12s trunk passed
          0 mvndep 0m 13s Maven dependency ordering for patch
          +1 mvninstall 8m 20s the patch passed
          +1 compile 6m 36s the patch passed
          -1 cc 15m 9s root generated 1 new + 10 unchanged - 1 fixed = 11 total (was 11)
          +1 cc 6m 36s the patch passed
          +1 javac 6m 36s the patch passed
          +1 mvnsite 8m 32s the patch passed
          +1 mvneclipse 0m 34s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 5m 44s the patch passed
          -1 unit 16m 45s root in the patch failed.
          +1 asflicense 0m 19s Patch does not generate ASF License warnings.
          77m 10s



          Reason Tests
          Failed junit tests hadoop.security.authentication.util.TestZKSignerSecretProvider



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806152/YARN-5121.01.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux 27ba6fa973d6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9a31e5d
          Default Java 1.8.0_91
          cc root: https://builds.apache.org/job/PreCommit-YARN-Build/11685/artifact/patchprocess/diff-compile-cc-root.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11685/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11685/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11685/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11685/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 13s Maven dependency ordering for branch +1 mvninstall 6m 10s trunk passed +1 compile 6m 40s trunk passed +1 mvnsite 9m 4s trunk passed +1 mvneclipse 0m 41s trunk passed +1 javadoc 6m 12s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 8m 20s the patch passed +1 compile 6m 36s the patch passed -1 cc 15m 9s root generated 1 new + 10 unchanged - 1 fixed = 11 total (was 11) +1 cc 6m 36s the patch passed +1 javac 6m 36s the patch passed +1 mvnsite 8m 32s the patch passed +1 mvneclipse 0m 34s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 5m 44s the patch passed -1 unit 16m 45s root in the patch failed. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 77m 10s Reason Tests Failed junit tests hadoop.security.authentication.util.TestZKSignerSecretProvider Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806152/YARN-5121.01.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux 27ba6fa973d6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9a31e5d Default Java 1.8.0_91 cc root: https://builds.apache.org/job/PreCommit-YARN-Build/11685/artifact/patchprocess/diff-compile-cc-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11685/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11685/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11685/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11685/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          -02:

          • rebased
          Show
          aw Allen Wittenauer added a comment - -02: rebased
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 35s Maven dependency ordering for branch
          +1 mvninstall 6m 28s trunk passed
          +1 compile 6m 14s trunk passed
          +1 mvnsite 8m 5s trunk passed
          +1 mvneclipse 0m 35s trunk passed
          -1 javadoc 3m 53s root in trunk failed.
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 6m 22s the patch passed
          +1 compile 6m 11s the patch passed
          +1 cc 6m 11s the patch passed
          +1 javac 6m 11s the patch passed
          +1 mvnsite 8m 3s the patch passed
          +1 mvneclipse 0m 33s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          -1 javadoc 3m 58s root in the patch failed.
          -1 unit 110m 9s root in the patch failed.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          162m 32s



          Reason Tests
          Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
            hadoop.hdfs.TestEncryptedTransfer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806372/YARN-5121.02.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux 69d0fca6719b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 77202fa
          Default Java 1.8.0_91
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/branch-javadoc-root.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/patch-javadoc-root.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11710/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/11710/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 35s Maven dependency ordering for branch +1 mvninstall 6m 28s trunk passed +1 compile 6m 14s trunk passed +1 mvnsite 8m 5s trunk passed +1 mvneclipse 0m 35s trunk passed -1 javadoc 3m 53s root in trunk failed. 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 6m 22s the patch passed +1 compile 6m 11s the patch passed +1 cc 6m 11s the patch passed +1 javac 6m 11s the patch passed +1 mvnsite 8m 3s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. -1 javadoc 3m 58s root in the patch failed. -1 unit 110m 9s root in the patch failed. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 162m 32s Reason Tests Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs   hadoop.hdfs.TestEncryptedTransfer Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806372/YARN-5121.02.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux 69d0fca6719b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 77202fa Default Java 1.8.0_91 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/branch-javadoc-root.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/patch-javadoc-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/11710/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11710/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/11710/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          Raising this to a blocker since it prevents the jenkins runs on the mac mini from working for yarn when using qbt.

          Show
          aw Allen Wittenauer added a comment - Raising this to a blocker since it prevents the jenkins runs on the mac mini from working for yarn when using qbt.
          Hide
          aw Allen Wittenauer added a comment -

          -03:

          • rebase again
          Show
          aw Allen Wittenauer added a comment - -03: rebase again
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 26s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 6m 32s trunk passed
          +1 compile 7m 11s trunk passed
          +1 mvnsite 8m 32s trunk passed
          +1 mvneclipse 0m 52s trunk passed
          +1 javadoc 4m 42s trunk passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 6m 42s the patch passed
          +1 compile 6m 33s the patch passed
          +1 cc 6m 33s the patch passed
          +1 javac 6m 33s the patch passed
          +1 mvnsite 8m 21s the patch passed
          +1 mvneclipse 0m 51s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 5m 26s the patch passed
          -1 unit 102m 32s root in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          160m 6s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.server.namenode.TestEditLog



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e2f6409
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812703/YARN-5121.03.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux db3f2f253991 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 99e5dd6
          Default Java 1.8.0_91
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12116/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12116/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12116/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12116/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 6m 32s trunk passed +1 compile 7m 11s trunk passed +1 mvnsite 8m 32s trunk passed +1 mvneclipse 0m 52s trunk passed +1 javadoc 4m 42s trunk passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 6m 42s the patch passed +1 compile 6m 33s the patch passed +1 cc 6m 33s the patch passed +1 javac 6m 33s the patch passed +1 mvnsite 8m 21s the patch passed +1 mvneclipse 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 5m 26s the patch passed -1 unit 102m 32s root in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 160m 6s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.TestEditLog Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12812703/YARN-5121.03.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux db3f2f253991 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 99e5dd6 Default Java 1.8.0_91 unit https://builds.apache.org/job/PreCommit-YARN-Build/12116/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12116/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12116/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/12116/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          +1 overall, though I haven't tested it on multiple platforms. Thanks for also updating the L&N

          Minor:

          • Leftover debug stmt in configuration.c?
            +    fprintf(stderr, "fn=%s\n",file_name);
                 strncpy(strrchr(buffer, '/') + 1, file_name, EXECUTOR_PATH_MAX);
                 real_fname = buffer;
            +    fprintf(stderr, "real_fname=%s\n",real_fname);
            
          • In container-executor.c, should "Error signalling process group %d with signal %d - %s\n" go to LOGFILE instead of stderr?
          • -0 on the whitespace fixes... I'd prefer to keep the history, but the patch touches enough code that it may be worthwhile.
          Show
          chris.douglas Chris Douglas added a comment - +1 overall, though I haven't tested it on multiple platforms. Thanks for also updating the L&N Minor: Leftover debug stmt in configuration.c ? + fprintf(stderr, "fn=%s\n",file_name); strncpy(strrchr(buffer, '/') + 1, file_name, EXECUTOR_PATH_MAX); real_fname = buffer; + fprintf(stderr, "real_fname=%s\n",real_fname); In container-executor.c , should "Error signalling process group %d with signal %d - %s\n" go to LOGFILE instead of stderr? -0 on the whitespace fixes... I'd prefer to keep the history, but the patch touches enough code that it may be worthwhile.
          Hide
          aw Allen Wittenauer added a comment -

          This will take some more work. I'll hack on it when I get back. Thanks for the feedback.

          Show
          aw Allen Wittenauer added a comment - This will take some more work. I'll hack on it when I get back. Thanks for the feedback.
          Hide
          aw Allen Wittenauer added a comment - - edited

          -04:

          • rebase
          • avoid argv[0] and use OS specific facilities where available/known
          • clean up some left over debug bits
          Show
          aw Allen Wittenauer added a comment - - edited -04: rebase avoid argv [0] and use OS specific facilities where available/known clean up some left over debug bits
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 7m 22s trunk passed
          +1 compile 7m 15s trunk passed
          +1 mvnsite 9m 26s trunk passed
          +1 mvneclipse 1m 7s trunk passed
          +1 javadoc 4m 42s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 8m 34s the patch passed
          -1 compile 3m 42s root in the patch failed.
          -1 cc 3m 42s root in the patch failed.
          -1 javac 3m 42s root in the patch failed.
          +1 mvnsite 11m 5s the patch passed
          +1 mvneclipse 1m 4s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 5m 40s the patch passed
          -1 unit 20m 47s root in the patch failed.
          -1 asflicense 0m 20s The patch generated 2 ASF License warnings.
          82m 24s



          Reason Tests
          Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
            hadoop.ipc.TestIPC
          Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820770/YARN-5121.04.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux 300475205a0c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 26de4f0
          Default Java 1.8.0_101
          compile https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-compile-root.txt
          cc https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-compile-root.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12543/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12543/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 7m 22s trunk passed +1 compile 7m 15s trunk passed +1 mvnsite 9m 26s trunk passed +1 mvneclipse 1m 7s trunk passed +1 javadoc 4m 42s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 8m 34s the patch passed -1 compile 3m 42s root in the patch failed. -1 cc 3m 42s root in the patch failed. -1 javac 3m 42s root in the patch failed. +1 mvnsite 11m 5s the patch passed +1 mvneclipse 1m 4s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 5m 40s the patch passed -1 unit 20m 47s root in the patch failed. -1 asflicense 0m 20s The patch generated 2 ASF License warnings. 82m 24s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.ipc.TestIPC Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820770/YARN-5121.04.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux 300475205a0c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 26de4f0 Default Java 1.8.0_101 compile https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-compile-root.txt cc https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-compile-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12543/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12543/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/12543/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          -05:

          • fix the compile error that for some reason didn't show up on my machine ?
          Show
          aw Allen Wittenauer added a comment - -05: fix the compile error that for some reason didn't show up on my machine ?
          Hide
          aw Allen Wittenauer added a comment -

          Now I'm seeing different compile errors on linux. Argh. Pulling back -05.

          Show
          aw Allen Wittenauer added a comment - Now I'm seeing different compile errors on linux. Argh. Pulling back -05.
          Hide
          aw Allen Wittenauer added a comment - - edited

          -06:

          • ok, let's try this again.
          • fixed some gcc warning as well
          Show
          aw Allen Wittenauer added a comment - - edited -06: ok, let's try this again. fixed some gcc warning as well
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 14s Maven dependency ordering for branch
          +1 mvninstall 7m 12s trunk passed
          +1 compile 7m 22s trunk passed
          +1 mvnsite 9m 51s trunk passed
          +1 mvneclipse 2m 36s trunk passed
          +1 javadoc 4m 58s trunk passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 7m 28s the patch passed
          +1 compile 7m 27s the patch passed
          -1 cc 7m 27s root generated 1 new + 8 unchanged - 2 fixed = 9 total (was 10)
          +1 javac 7m 27s the patch passed
          +1 mvnsite 9m 46s the patch passed
          +1 mvneclipse 1m 1s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 4m 56s the patch passed
          -1 unit 112m 2s root in the patch failed.
          -1 asflicense 0m 23s The patch generated 2 ASF License warnings.
          176m 25s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestErasureCodeBenchmarkThroughput
            hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
            hadoop.yarn.server.nodemanager.TestDirectoryCollection
            hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820790/YARN-5121.06.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux 489bb4ed891f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a3d0cba
          Default Java 1.8.0_101
          cc https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/diff-compile-cc-root.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12547/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12547/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 7m 12s trunk passed +1 compile 7m 22s trunk passed +1 mvnsite 9m 51s trunk passed +1 mvneclipse 2m 36s trunk passed +1 javadoc 4m 58s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 7m 28s the patch passed +1 compile 7m 27s the patch passed -1 cc 7m 27s root generated 1 new + 8 unchanged - 2 fixed = 9 total (was 10) +1 javac 7m 27s the patch passed +1 mvnsite 9m 46s the patch passed +1 mvneclipse 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 4m 56s the patch passed -1 unit 112m 2s root in the patch failed. -1 asflicense 0m 23s The patch generated 2 ASF License warnings. 176m 25s Reason Tests Failed junit tests hadoop.hdfs.TestErasureCodeBenchmarkThroughput   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.yarn.server.nodemanager.TestDirectoryCollection   hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820790/YARN-5121.06.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux 489bb4ed891f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a3d0cba Default Java 1.8.0_101 cc https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/diff-compile-cc-root.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12547/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12547/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/12547/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          asf license issues are unrelated. unit test failures are unrelated. and the new compiler warning is trivial to fix on commit.

          Show
          aw Allen Wittenauer added a comment - asf license issues are unrelated. unit test failures are unrelated. and the new compiler warning is trivial to fix on commit.
          Hide
          cnauroth Chris Nauroth added a comment -

          Allen Wittenauer, thank you for this patch. I have confirmed a successful full build and run of test-container-executor on OS X and Linux.

          Just a few questions:

          For hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/{fstatat|openat|unlinkat}.h:

          I just want to double-check with you that the fchmodat.h and fdopendir.h implementations are not BSD-licensed, and that's why they're not listed in LICENSE.txt and instead have an Apache license header. Is that correct?

            fprintf(stderr,"ret = %s\n", ret);
          

          Chris D mentioned previously that this might have been a leftover from debugging. Did you intend to keep it, or should we drop it?

          char* get_executable() {
           return __get_exec_readproc("/proc/self/path/a.out");
          }
          

          Please check the indentation on the return statement.

          Is "/proc/self/path/a.out" correct? The /proc/self part makes sense to me, but the rest of it surprised me. Is that a.out like the default gcc binary output path? I have nearly zero experience with Solaris, so I trust your knowledge here.

          Show
          cnauroth Chris Nauroth added a comment - Allen Wittenauer , thank you for this patch. I have confirmed a successful full build and run of test-container-executor on OS X and Linux. Just a few questions: For hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/{fstatat|openat|unlinkat}.h: I just want to double-check with you that the fchmodat.h and fdopendir.h implementations are not BSD-licensed, and that's why they're not listed in LICENSE.txt and instead have an Apache license header. Is that correct? fprintf(stderr, "ret = %s\n" , ret); Chris D mentioned previously that this might have been a leftover from debugging. Did you intend to keep it, or should we drop it? char * get_executable() { return __get_exec_readproc( "/proc/self/path/a.out" ); } Please check the indentation on the return statement. Is "/proc/self/path/a.out" correct? The /proc/self part makes sense to me, but the rest of it surprised me. Is that a.out like the default gcc binary output path? I have nearly zero experience with Solaris, so I trust your knowledge here.
          Hide
          aw Allen Wittenauer added a comment -

          I just want to double-check with you that the fchmodat.h and fdopendir.h implementations are not BSD-licensed, and that's why they're not listed in LICENSE.txt and instead have an Apache license header. Is that correct?

          I wrote them based upon the other functions' implementations so I put an ASF license on them.

          Chris D mentioned previously that this might have been a leftover from debugging. Did you intend to keep it.

          I did remove some other debugging code, but that one I thought was useful due to aggressive use of ternary operators. (UGH! Yes, the code is compact, but nearly unreadable, when nested! I probably should have rewrote them too but...)

          Please check the indentation on the return statement.

          Argh. Yes, I'll fix. The formatting of the C code is pretty awful and definitely caused me issues. haha.

          Is "/proc/self/path/a.out" correct? ... Is that a.out like the default gcc binary output path?

          a.out was the file name before gcc existed... Anyway, here's some backing evidence:

          http://docs.oracle.com/cd/E23824_01/html/821-1473/proc-4.html#

          Solaris' proc works a bit differently than what one might be used to under Linux. The keys here are /proc/pid/object and /proc/pid/path. object gives you access to any mapped or page data that came from file system objects. path contains symbolic links to all open files and the source files of the content of the object dir. object/a.out is the executable. Therefore path/a.out is a symbolic link to the executable itself. With that context, it probably makes a lot more sense, since there are two ways to get to the data, depending upon your needs.

          Anyway, from my home machine (skipping object, since the file names are present in path):

          sunos/i386 ryoko$ pwd
          /proc/6956/path
          
          sunos/i386 ryoko$ ls -l
          total 0
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 0 -> /dev/pts/3
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 1 -> /dev/pts/3
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 2 -> /dev/pts/3
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 255 -> /dev/pts/3
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 3 -> /var/run/ldap_cache_door
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 a.out -> /usr/bin/bash
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 cwd -> /home/allenw
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 root -> /
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.11300 -> /usr/lib/mps/libnspr4.so
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18905 -> /lib/libmd.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18918 -> /lib/libgen.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18920 -> /lib/libsocket.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18942 -> /lib/libresolv.so.2
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18950 -> /lib/nss_files.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18956 -> /lib/libmp.so.2
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18957 -> /lib/nss_dns.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18961 -> /lib/libpthread.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.18964 -> /lib/libnsl.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.6054 -> /lib/ld.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.6085 -> /lib/libcurses.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.6088 -> /lib/libdl.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.9190 -> /usr/lib/libc/libc_hwcap1.so.1
          lrwxrwxrwx   1 allenw   users          0 Jun 21 08:05 zfs.122.65594.9233 -> /usr/lib/libldap.so.5
          

          You'll note that a.out points to the bash executable.

          (This whole conversation is rather timely, given that Roger Faulkner just passed away recently.)

          I'll do a quick update and post a new patch. Thanks for the review!

          Show
          aw Allen Wittenauer added a comment - I just want to double-check with you that the fchmodat.h and fdopendir.h implementations are not BSD-licensed, and that's why they're not listed in LICENSE.txt and instead have an Apache license header. Is that correct? I wrote them based upon the other functions' implementations so I put an ASF license on them. Chris D mentioned previously that this might have been a leftover from debugging. Did you intend to keep it. I did remove some other debugging code, but that one I thought was useful due to aggressive use of ternary operators. (UGH! Yes, the code is compact, but nearly unreadable, when nested! I probably should have rewrote them too but...) Please check the indentation on the return statement. Argh. Yes, I'll fix. The formatting of the C code is pretty awful and definitely caused me issues. haha. Is "/proc/self/path/a.out" correct? ... Is that a.out like the default gcc binary output path? a.out was the file name before gcc existed... Anyway, here's some backing evidence: http://docs.oracle.com/cd/E23824_01/html/821-1473/proc-4.html# Solaris' proc works a bit differently than what one might be used to under Linux. The keys here are /proc/pid/object and /proc/pid/path. object gives you access to any mapped or page data that came from file system objects. path contains symbolic links to all open files and the source files of the content of the object dir. object/a.out is the executable. Therefore path/a.out is a symbolic link to the executable itself. With that context, it probably makes a lot more sense, since there are two ways to get to the data, depending upon your needs. Anyway, from my home machine (skipping object, since the file names are present in path): sunos/i386 ryoko$ pwd /proc/6956/path sunos/i386 ryoko$ ls -l total 0 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 0 -> /dev/pts/3 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 1 -> /dev/pts/3 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 2 -> /dev/pts/3 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 255 -> /dev/pts/3 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 3 -> / var /run/ldap_cache_door lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 a.out -> /usr/bin/bash lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 cwd -> /home/allenw lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 root -> / lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.11300 -> /usr/lib/mps/libnspr4.so lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18905 -> /lib/libmd.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18918 -> /lib/libgen.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18920 -> /lib/libsocket.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18942 -> /lib/libresolv.so.2 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18950 -> /lib/nss_files.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18956 -> /lib/libmp.so.2 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18957 -> /lib/nss_dns.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18961 -> /lib/libpthread.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.18964 -> /lib/libnsl.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.6054 -> /lib/ld.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.6085 -> /lib/libcurses.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.6088 -> /lib/libdl.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.9190 -> /usr/lib/libc/libc_hwcap1.so.1 lrwxrwxrwx 1 allenw users 0 Jun 21 08:05 zfs.122.65594.9233 -> /usr/lib/libldap.so.5 You'll note that a.out points to the bash executable. (This whole conversation is rather timely, given that Roger Faulkner just passed away recently.) I'll do a quick update and post a new patch. Thanks for the review!
          Hide
          cnauroth Chris Nauroth added a comment -

          Thanks for the detailed explanation. It's all clear to me now. I expect this will be ready to commit after your next revision to fix the few remaining nitpicks. That next revision can fix the one remaining compiler warning too.

          Chris Douglas, let us know if you have any more feedback. If not, then I would likely +1 and commit soon.

          (This whole conversation is rather timely, given that Roger Faulkner just passed away recently.)

          I did not know the name before, but I just read an "In Memoriam" article. Thank you, Roger.

          Show
          cnauroth Chris Nauroth added a comment - Thanks for the detailed explanation. It's all clear to me now. I expect this will be ready to commit after your next revision to fix the few remaining nitpicks. That next revision can fix the one remaining compiler warning too. Chris Douglas , let us know if you have any more feedback. If not, then I would likely +1 and commit soon. (This whole conversation is rather timely, given that Roger Faulkner just passed away recently.) I did not know the name before, but I just read an "In Memoriam" article. Thank you, Roger.
          Hide
          aw Allen Wittenauer added a comment -

          -07:

          • address the feedback
          • fix two more spurious gcc warnings
          Show
          aw Allen Wittenauer added a comment - -07: address the feedback fix two more spurious gcc warnings
          Hide
          chris.douglas Chris Douglas added a comment -

          +1 from me. Thanks, Allen for the patch and ChrisN for review.

          I did remove some other debugging code, but that one I thought was useful due to aggressive use of ternary operators

          I haven't looked at the context, but if ret can never be null in that case (real_fname is never null?), then the tenary operator is redundant. If it can be null, then the new debug stmt can cause a segfault before it prints? Nit-picking in any case.

          Show
          chris.douglas Chris Douglas added a comment - +1 from me. Thanks, Allen for the patch and ChrisN for review. I did remove some other debugging code, but that one I thought was useful due to aggressive use of ternary operators I haven't looked at the context, but if ret can never be null in that case ( real_fname is never null?), then the tenary operator is redundant. If it can be null, then the new debug stmt can cause a segfault before it prints? Nit-picking in any case.
          Hide
          cnauroth Chris Nauroth added a comment -

          Allen, sorry, I just spotted one more thing. Would you please check for NULL returns from the malloc calls in __get_exec_readproc and the OS X implementation of get_executable?

          Show
          cnauroth Chris Nauroth added a comment - Allen, sorry, I just spotted one more thing. Would you please check for NULL returns from the malloc calls in __get_exec_readproc and the OS X implementation of get_executable ?
          Hide
          aw Allen Wittenauer added a comment -

          if ret can never be null in that case (real_fname is never null?), then the tenary operator is redundant. If it can be null, then the new debug stmt can cause a segfault before it prints? Nit-picking in any case.

          IIRC, I think the only time that ret can ever be null is if the mallocs inside realpath/canonicalize_file_name fail. My hunch is yes, that ternary could probably go away and there should be some more safety. I didn't spend much time digging into it though. There's a lot of code like that all over the place and it's definitely a much bigger project to fix those problems.

          So, yes, there's definitely a risk that the debug will cause a segfault but the code isn't enabled by default and someone enabling it is likely trying to figure out what the heck is going on anyway haha.

          Would you please check for NULL returns from the malloc calls in __get_exec_readproc and the OS X implementation of get_executable?

          Sure. Good catch. They were missing in the original, likely because realpath will fail and you'll exit out there instead. But it'd definitely be better to have a real check with a specific error. (That also means flushing the log. We really should be using atexit(). This whole thing needs a rewrite.)

          Show
          aw Allen Wittenauer added a comment - if ret can never be null in that case (real_fname is never null?), then the tenary operator is redundant. If it can be null, then the new debug stmt can cause a segfault before it prints? Nit-picking in any case. IIRC, I think the only time that ret can ever be null is if the mallocs inside realpath/canonicalize_file_name fail. My hunch is yes, that ternary could probably go away and there should be some more safety. I didn't spend much time digging into it though. There's a lot of code like that all over the place and it's definitely a much bigger project to fix those problems. So, yes, there's definitely a risk that the debug will cause a segfault but the code isn't enabled by default and someone enabling it is likely trying to figure out what the heck is going on anyway haha. Would you please check for NULL returns from the malloc calls in __get_exec_readproc and the OS X implementation of get_executable? Sure. Good catch. They were missing in the original, likely because realpath will fail and you'll exit out there instead. But it'd definitely be better to have a real check with a specific error. (That also means flushing the log. We really should be using atexit(). This whole thing needs a rewrite.)
          Hide
          aw Allen Wittenauer added a comment -

          Oh right... we can't call flush_and_close there because the function isn't shared with the rest of the source code. Just shoot me now.

          Anyway, here's -08.

          Show
          aw Allen Wittenauer added a comment - Oh right... we can't call flush_and_close there because the function isn't shared with the rest of the source code. Just shoot me now. Anyway, here's -08.
          Hide
          aw Allen Wittenauer added a comment -

          -08:

          • add malloc checks rather than letting realpath throw the error
          Show
          aw Allen Wittenauer added a comment - -08: add malloc checks rather than letting realpath throw the error
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 1m 12s Maven dependency ordering for branch
          +1 mvninstall 6m 39s trunk passed
          +1 compile 6m 56s trunk passed
          +1 mvnsite 10m 56s trunk passed
          +1 mvneclipse 1m 5s trunk passed
          +1 javadoc 5m 26s trunk passed
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 8m 33s the patch passed
          +1 compile 7m 58s the patch passed
          +1 cc 7m 58s root generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10)
          +1 javac 7m 58s the patch passed
          +1 mvnsite 9m 28s the patch passed
          +1 mvneclipse 1m 0s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 4m 48s the patch passed
          -1 unit 118m 15s root in the patch failed.
          -1 asflicense 0m 24s The patch generated 2 ASF License warnings.
          183m 58s



          Reason Tests
          Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager
            hadoop.yarn.server.nodemanager.TestDirectoryCollection



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821118/YARN-5121.07.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux 95046fd3e110 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 95f2b98
          Default Java 1.8.0_101
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12571/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12571/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 12s Maven dependency ordering for branch +1 mvninstall 6m 39s trunk passed +1 compile 6m 56s trunk passed +1 mvnsite 10m 56s trunk passed +1 mvneclipse 1m 5s trunk passed +1 javadoc 5m 26s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 8m 33s the patch passed +1 compile 7m 58s the patch passed +1 cc 7m 58s root generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10) +1 javac 7m 58s the patch passed +1 mvnsite 9m 28s the patch passed +1 mvneclipse 1m 0s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 4m 48s the patch passed -1 unit 118m 15s root in the patch failed. -1 asflicense 0m 24s The patch generated 2 ASF License warnings. 183m 58s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.namenode.TestEditLog   hadoop.yarn.server.nodemanager.containermanager.queuing.TestQueuingContainerManager   hadoop.yarn.server.nodemanager.TestDirectoryCollection Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821118/YARN-5121.07.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux 95046fd3e110 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 95f2b98 Default Java 1.8.0_101 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12571/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12571/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/12571/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 1m 8s Maven dependency ordering for branch
          +1 mvninstall 7m 29s trunk passed
          +1 compile 7m 52s trunk passed
          +1 mvnsite 11m 28s trunk passed
          +1 mvneclipse 2m 47s trunk passed
          +1 javadoc 5m 36s trunk passed
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 8m 54s the patch passed
          +1 compile 8m 17s the patch passed
          +1 cc 8m 17s root generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10)
          +1 javac 8m 17s the patch passed
          +1 mvnsite 10m 45s the patch passed
          +1 mvneclipse 1m 8s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 5m 53s the patch passed
          -1 unit 109m 48s root in the patch failed.
          -1 asflicense 0m 23s The patch generated 2 ASF License warnings.
          182m 52s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.yarn.server.nodemanager.TestDirectoryCollection



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821127/YARN-5121.08.patch
          JIRA Issue YARN-5121
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc
          uname Linux f6917c7b1155 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 95f2b98
          Default Java 1.8.0_101
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/patch-unit-root.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12573/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: .
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12573/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 8s Maven dependency ordering for branch +1 mvninstall 7m 29s trunk passed +1 compile 7m 52s trunk passed +1 mvnsite 11m 28s trunk passed +1 mvneclipse 2m 47s trunk passed +1 javadoc 5m 36s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 8m 54s the patch passed +1 compile 8m 17s the patch passed +1 cc 8m 17s root generated 0 new + 7 unchanged - 3 fixed = 7 total (was 10) +1 javac 8m 17s the patch passed +1 mvnsite 10m 45s the patch passed +1 mvneclipse 1m 8s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 5m 53s the patch passed -1 unit 109m 48s root in the patch failed. -1 asflicense 0m 23s The patch generated 2 ASF License warnings. 182m 52s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer   hadoop.hdfs.server.namenode.TestEditLog   hadoop.yarn.server.nodemanager.TestDirectoryCollection Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821127/YARN-5121.08.patch JIRA Issue YARN-5121 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml cc uname Linux f6917c7b1155 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 95f2b98 Default Java 1.8.0_101 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/patch-unit-root.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12573/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/12573/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . U: . Console output https://builds.apache.org/job/PreCommit-YARN-Build/12573/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          FYI:

          For the OSS BSDs, apparently the answer is to use a sysctl call against KERN_PROC_PATHNAME. I'll do those as a follow-up JIRA so that we can get this one in first.

          Show
          aw Allen Wittenauer added a comment - FYI: For the OSS BSDs, apparently the answer is to use a sysctl call against KERN_PROC_PATHNAME. I'll do those as a follow-up JIRA so that we can get this one in first.
          Hide
          cnauroth Chris Nauroth added a comment -

          I verified patch 08 once again on OS X and Linux. +1. I committed it to trunk. Allen, thank you for the patch. Chris D, thank you for your code review.

          Show
          cnauroth Chris Nauroth added a comment - I verified patch 08 once again on OS X and Linux. +1. I committed it to trunk. Allen, thank you for the patch. Chris D, thank you for your code review.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10182 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10182/)
          YARN-5121. fix some container-executor portability issues. Contributed (cnauroth: rev ef501b1a0b4c34a2cc43eb082d1c2364684cd7f1)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/fchmodat.h
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/unlinkat.h
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/fdopendir.h
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/get_executable.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/openat.h
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.h
          • LICENSE.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/fstatat.h
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10182 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10182/ ) YARN-5121 . fix some container-executor portability issues. Contributed (cnauroth: rev ef501b1a0b4c34a2cc43eb082d1c2364684cd7f1) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/fchmodat.h hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/unlinkat.h hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/fdopendir.h hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/get_executable.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/openat.h hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.h LICENSE.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/compat/fstatat.h
          Hide
          aw Allen Wittenauer added a comment -

          Thanks for the reviews and for helping to get this in guys!

          Show
          aw Allen Wittenauer added a comment - Thanks for the reviews and for helping to get this in guys!
          Hide
          shv Konstantin Shvachko added a comment -

          Committed this to branch-2.7. Thank you Akira Ajisaka for backport.
          Could you please attach the final patch for branch-2.7 from YARN-6698 here.

          Show
          shv Konstantin Shvachko added a comment - Committed this to branch-2.7. Thank you Akira Ajisaka for backport. Could you please attach the final patch for branch-2.7 from YARN-6698 here.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Attaching the patch for branch-2.7. Thanks.

          Show
          ajisakaa Akira Ajisaka added a comment - Attaching the patch for branch-2.7. Thanks.

            People

            • Assignee:
              aw Allen Wittenauer
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development