Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4579

Allow DefaultContainerExecutor container log directory permissions to be configurable

    Details

    • Hadoop Flags:
      Reviewed

      Description

      By default, container directory permissions are hardcoded to this member in DefaultContainerExecutor:

      static final short LOGDIR_PERM = (short)0710;

      There are some cases where less restrictive permissions are desired. Make this configurable.

      1. YARN-4579.001.patch
        6 kB
        Ray Chiang
      2. YARN-4579.002.patch
        7 kB
        Ray Chiang
      3. YARN-4579.003.patch
        7 kB
        Ray Chiang
      4. YARN-4579.004.patch
        8 kB
        Ray Chiang
      5. YARN-4579.005.patch
        8 kB
        Ray Chiang
      6. YARN-4579.006.patch
        8 kB
        Ray Chiang

        Activity

        Hide
        rchiang Ray Chiang added a comment -

        Initial version.

        Show
        rchiang Ray Chiang added a comment - Initial version.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 47s trunk passed
        +1 compile 1m 47s trunk passed with JDK v1.8.0_66
        +1 compile 2m 6s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 2m 15s trunk passed
        +1 javadoc 0m 58s trunk passed with JDK v1.8.0_66
        +1 javadoc 3m 8s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 48s the patch passed
        +1 compile 1m 42s the patch passed with JDK v1.8.0_66
        +1 javac 1m 42s the patch passed
        +1 compile 2m 4s the patch passed with JDK v1.7.0_91
        +1 javac 2m 4s the patch passed
        -1 checkstyle 0m 29s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237).
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 21s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 25s the patch passed
        +1 javadoc 0m 51s the patch passed with JDK v1.8.0_66
        +1 javadoc 3m 9s the patch passed with JDK v1.7.0_91
        -1 unit 0m 20s hadoop-yarn-api in the patch failed with JDK v1.8.0_66.
        +1 unit 8m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        -1 unit 0m 23s hadoop-yarn-api in the patch failed with JDK v1.7.0_91.
        +1 unit 9m 14s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        52m 47s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields
        JDK v1.7.0_91 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781681/YARN-4579.001.patch
        JIRA Issue YARN-4579
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c159b2d00d3f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / de37f37
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10233/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10233/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 47s trunk passed +1 compile 1m 47s trunk passed with JDK v1.8.0_66 +1 compile 2m 6s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 30s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 2m 15s trunk passed +1 javadoc 0m 58s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 8s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 48s the patch passed +1 compile 1m 42s the patch passed with JDK v1.8.0_66 +1 javac 1m 42s the patch passed +1 compile 2m 4s the patch passed with JDK v1.7.0_91 +1 javac 2m 4s the patch passed -1 checkstyle 0m 29s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237). +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 21s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 25s the patch passed +1 javadoc 0m 51s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 9s the patch passed with JDK v1.7.0_91 -1 unit 0m 20s hadoop-yarn-api in the patch failed with JDK v1.8.0_66. +1 unit 8m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. -1 unit 0m 23s hadoop-yarn-api in the patch failed with JDK v1.7.0_91. +1 unit 9m 14s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 52m 47s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields JDK v1.7.0_91 Failed junit tests hadoop.yarn.conf.TestYarnConfigurationFields Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781681/YARN-4579.001.patch JIRA Issue YARN-4579 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c159b2d00d3f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / de37f37 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/10233/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10233/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10233/console This message was automatically generated.
        Hide
        rchiang Ray Chiang added a comment -

        Fix checkstyle and XML unit test error.

        Show
        rchiang Ray Chiang added a comment - Fix checkstyle and XML unit test error.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 14s trunk passed
        +1 compile 2m 2s trunk passed with JDK v1.8.0_66
        +1 compile 2m 17s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 31s trunk passed
        +1 mvnsite 1m 35s trunk passed
        +1 mvneclipse 0m 40s trunk passed
        +1 findbugs 3m 54s trunk passed
        +1 javadoc 1m 31s trunk passed with JDK v1.8.0_66
        +1 javadoc 3m 52s trunk passed with JDK v1.7.0_91
        -1 mvninstall 0m 16s hadoop-yarn-server-nodemanager in the patch failed.
        -1 compile 0m 50s hadoop-yarn in the patch failed with JDK v1.8.0_66.
        -1 javac 0m 50s hadoop-yarn in the patch failed with JDK v1.8.0_66.
        -1 compile 0m 57s hadoop-yarn in the patch failed with JDK v1.7.0_91.
        -1 javac 0m 57s hadoop-yarn in the patch failed with JDK v1.7.0_91.
        -1 checkstyle 0m 32s Patch generated 1 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 234).
        -1 mvnsite 0m 17s hadoop-yarn-server-nodemanager in the patch failed.
        +1 mvneclipse 0m 32s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        -1 findbugs 0m 16s hadoop-yarn-server-nodemanager in the patch failed.
        +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66
        +1 javadoc 3m 45s the patch passed with JDK v1.7.0_91
        +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
        +1 unit 2m 1s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        -1 unit 0m 15s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66.
        +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
        +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        -1 unit 0m 16s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        45m 43s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781906/YARN-4579.002.patch
        JIRA Issue YARN-4579
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 67209b8eebbd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 126705f
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        compile https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt
        javac https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt
        compile https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt
        javac https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10252/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10252/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 14s trunk passed +1 compile 2m 2s trunk passed with JDK v1.8.0_66 +1 compile 2m 17s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 35s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 3m 54s trunk passed +1 javadoc 1m 31s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 52s trunk passed with JDK v1.7.0_91 -1 mvninstall 0m 16s hadoop-yarn-server-nodemanager in the patch failed. -1 compile 0m 50s hadoop-yarn in the patch failed with JDK v1.8.0_66. -1 javac 0m 50s hadoop-yarn in the patch failed with JDK v1.8.0_66. -1 compile 0m 57s hadoop-yarn in the patch failed with JDK v1.7.0_91. -1 javac 0m 57s hadoop-yarn in the patch failed with JDK v1.7.0_91. -1 checkstyle 0m 32s Patch generated 1 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 234). -1 mvnsite 0m 17s hadoop-yarn-server-nodemanager in the patch failed. +1 mvneclipse 0m 32s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 0m 16s hadoop-yarn-server-nodemanager in the patch failed. +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 45s the patch passed with JDK v1.7.0_91 +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 2m 1s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 0m 15s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66. +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. -1 unit 0m 16s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 45m 43s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781906/YARN-4579.002.patch JIRA Issue YARN-4579 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 67209b8eebbd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 126705f Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn-jdk1.7.0_91.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10252/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10252/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10252/console This message was automatically generated.
        Hide
        rchiang Ray Chiang added a comment -

        Get patch from correct directory this time.

        Show
        rchiang Ray Chiang added a comment - Get patch from correct directory this time.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 11s trunk passed
        +1 compile 2m 21s trunk passed with JDK v1.8.0_66
        +1 compile 2m 5s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 29s trunk passed
        +1 mvneclipse 0m 45s trunk passed
        +1 findbugs 3m 38s trunk passed
        +1 javadoc 1m 33s trunk passed with JDK v1.8.0_66
        +1 javadoc 3m 50s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 14s the patch passed
        +1 compile 2m 10s the patch passed with JDK v1.8.0_66
        +1 javac 2m 10s the patch passed
        +1 compile 2m 10s the patch passed with JDK v1.7.0_91
        +1 javac 2m 10s the patch passed
        -1 checkstyle 0m 29s Patch generated 1 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 234).
        +1 mvnsite 1m 22s the patch passed
        +1 mvneclipse 0m 31s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 3m 53s the patch passed
        +1 javadoc 1m 33s the patch passed with JDK v1.8.0_66
        +1 javadoc 3m 36s the patch passed with JDK v1.7.0_91
        +1 unit 0m 21s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 56s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 47s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
        +1 unit 2m 12s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 9m 7s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        66m 9s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781927/YARN-4579.003.patch
        JIRA Issue YARN-4579
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 538592f30b9e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 126705f
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10253/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10253/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10253/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 11s trunk passed +1 compile 2m 21s trunk passed with JDK v1.8.0_66 +1 compile 2m 5s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 29s trunk passed +1 mvneclipse 0m 45s trunk passed +1 findbugs 3m 38s trunk passed +1 javadoc 1m 33s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 50s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 14s the patch passed +1 compile 2m 10s the patch passed with JDK v1.8.0_66 +1 javac 2m 10s the patch passed +1 compile 2m 10s the patch passed with JDK v1.7.0_91 +1 javac 2m 10s the patch passed -1 checkstyle 0m 29s Patch generated 1 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 234). +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 31s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 53s the patch passed +1 javadoc 1m 33s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 36s the patch passed with JDK v1.7.0_91 +1 unit 0m 21s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 1m 56s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 47s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 12s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 9m 7s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 66m 9s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781927/YARN-4579.003.patch JIRA Issue YARN-4579 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 538592f30b9e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 126705f Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10253/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10253/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10253/console This message was automatically generated.
        Hide
        rchiang Ray Chiang added a comment -

        RE: checkstyle

        File length already exceeded limit.

        Show
        rchiang Ray Chiang added a comment - RE: checkstyle File length already exceeded limit.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        There are some cases where less restrictive permissions are desired. Make this configurable.

        Can you elaborate on this?

        If the use-case has merits, this shouldn't be limited only to DefaultContainerExecutor? And depending on what it is, it may (or may not) apply to other dirs like local-dirs too.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - There are some cases where less restrictive permissions are desired. Make this configurable. Can you elaborate on this? If the use-case has merits, this shouldn't be limited only to DefaultContainerExecutor? And depending on what it is, it may (or may not) apply to other dirs like local-dirs too.
        Hide
        rkanter Robert Kanter added a comment -

        Given that the property name in the patch is yarn.nodemanager.log-dirs.permissions, it should be made to apply to all container executors. Though if we only want this for DCE (e.g. this could be a security concern for LCE?), then I think the property name should reflect that it's only for DCE.

        Show
        rkanter Robert Kanter added a comment - Given that the property name in the patch is yarn.nodemanager.log-dirs.permissions , it should be made to apply to all container executors. Though if we only want this for DCE (e.g. this could be a security concern for LCE?), then I think the property name should reflect that it's only for DCE.
        Hide
        rchiang Ray Chiang added a comment -

        I don't have all the specifics, but I have one request where they're using a third-party tool to pull data from the container logs. The tool can't run as user 'yarn' and the hardcoded directory permissions of 710 is preventing this tool/flow from working. I do agree it's a bit of a weird corner case, since I'd assume this would only apply to customers that aren't as concerned about security (at least with respect to logs).

        As for design, it looks like each subclass of ContainerExecutor has its own implementation (or inherited) of startLocalizer(). Are you thinking of generalizing the directory location/permissions/other requirements into LocalizerStartContext or did you have something else in mind?

        Show
        rchiang Ray Chiang added a comment - I don't have all the specifics, but I have one request where they're using a third-party tool to pull data from the container logs. The tool can't run as user 'yarn' and the hardcoded directory permissions of 710 is preventing this tool/flow from working. I do agree it's a bit of a weird corner case, since I'd assume this would only apply to customers that aren't as concerned about security (at least with respect to logs). As for design, it looks like each subclass of ContainerExecutor has its own implementation (or inherited) of startLocalizer(). Are you thinking of generalizing the directory location/permissions/other requirements into LocalizerStartContext or did you have something else in mind?
        Hide
        rchiang Ray Chiang added a comment -

        I'm okay with that, depending on the feedback I get from Vinod in the previous comment about making this feature available for all subclasses of ContainerExecutor vs. only DefaultContainerExecutor.

        Show
        rchiang Ray Chiang added a comment - I'm okay with that, depending on the feedback I get from Vinod in the previous comment about making this feature available for all subclasses of ContainerExecutor vs. only DefaultContainerExecutor.
        Hide
        rkanter Robert Kanter added a comment -

        Can you update the unit test to also use a non-default value and verify that it's actually using the configured value?

        Show
        rkanter Robert Kanter added a comment - Can you update the unit test to also use a non-default value and verify that it's actually using the configured value?
        Hide
        rchiang Ray Chiang added a comment -

        I would think that since the container log directory is the only one generated by YARN, so there could be useful information in there. The other directories (file cache, app cache, user directory) would be files the user could already have access to without even launching a job, so I would expect that permissions there would be less likely to need loosening.

        One follow up thought, based on Robert's feedback. Does it make sense to make it a DefaultContainerExecutor property only? For security reasons, it might make sense to give each ContainerExecutor subclass it's own property for container log directory permissions.

        If so, I can do this JIRA for DefaultContainerExecutor and do a follow up JIRA to refactor ContainerExecutor and it's subclasses for the other properties. I'd like a little more time to think on that.

        Show
        rchiang Ray Chiang added a comment - I would think that since the container log directory is the only one generated by YARN, so there could be useful information in there. The other directories (file cache, app cache, user directory) would be files the user could already have access to without even launching a job, so I would expect that permissions there would be less likely to need loosening. One follow up thought, based on Robert's feedback. Does it make sense to make it a DefaultContainerExecutor property only? For security reasons, it might make sense to give each ContainerExecutor subclass it's own property for container log directory permissions. If so, I can do this JIRA for DefaultContainerExecutor and do a follow up JIRA to refactor ContainerExecutor and it's subclasses for the other properties. I'd like a little more time to think on that.
        Hide
        rchiang Ray Chiang added a comment -

        Good idea. I will do that.

        Show
        rchiang Ray Chiang added a comment - Good idea. I will do that.
        Hide
        rchiang Ray Chiang added a comment -
        • Change property name to be DefaultContainerExecutor specific
        • Update the unit test to check a couple of permission strings

        We can move discussions about permissions configuration vs. security across ContainerExecutor subclasses to a separate JIRA.

        Show
        rchiang Ray Chiang added a comment - Change property name to be DefaultContainerExecutor specific Update the unit test to check a couple of permission strings We can move discussions about permissions configuration vs. security across ContainerExecutor subclasses to a separate JIRA.
        Hide
        rchiang Ray Chiang added a comment -

        Sorry Vinod Kumar Vavilapalli, I didn't realize that replying within the JIRA may not send a notification. I've re-quoted my earlier comments below:

        I don't have all the specifics, but I have one request where they're using a third-party tool to pull data from the container logs. The tool can't run as user 'yarn' and the hardcoded directory permissions of 710 is preventing this tool/flow from working. I do agree it's a bit of a weird corner case, since I'd assume this would only apply to customers that aren't as concerned about security (at least with respect to logs).

        As for design, it looks like each subclass of ContainerExecutor has its own implementation (or inherited) of startLocalizer(). Are you thinking of generalizing the directory location/permissions/other requirements into LocalizerStartContext or did you have something else in mind?

        I would think that since the container log directory is the only one generated by YARN, so there could be useful information in there. The other directories (file cache, app cache, user directory) would be files the user could already have access to without even launching a job, so I would expect that permissions there would be less likely to need loosening.

        One follow up thought, based on Robert's feedback. Does it make sense to make it a DefaultContainerExecutor property only? For security reasons, it might make sense to give each ContainerExecutor subclass it's own property for container log directory permissions. If so, I can do this JIRA for DefaultContainerExecutor and do a follow up JIRA to refactor ContainerExecutor and it's subclasses for the other properties. I'd like a little more time to think on that.

        Show
        rchiang Ray Chiang added a comment - Sorry Vinod Kumar Vavilapalli , I didn't realize that replying within the JIRA may not send a notification. I've re-quoted my earlier comments below: I don't have all the specifics, but I have one request where they're using a third-party tool to pull data from the container logs. The tool can't run as user 'yarn' and the hardcoded directory permissions of 710 is preventing this tool/flow from working. I do agree it's a bit of a weird corner case, since I'd assume this would only apply to customers that aren't as concerned about security (at least with respect to logs). As for design, it looks like each subclass of ContainerExecutor has its own implementation (or inherited) of startLocalizer(). Are you thinking of generalizing the directory location/permissions/other requirements into LocalizerStartContext or did you have something else in mind? I would think that since the container log directory is the only one generated by YARN, so there could be useful information in there. The other directories (file cache, app cache, user directory) would be files the user could already have access to without even launching a job, so I would expect that permissions there would be less likely to need loosening. One follow up thought, based on Robert's feedback. Does it make sense to make it a DefaultContainerExecutor property only? For security reasons, it might make sense to give each ContainerExecutor subclass it's own property for container log directory permissions. If so, I can do this JIRA for DefaultContainerExecutor and do a follow up JIRA to refactor ContainerExecutor and it's subclasses for the other properties. I'd like a little more time to think on that.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 40s Maven dependency ordering for branch
        +1 mvninstall 7m 48s trunk passed
        +1 compile 1m 53s trunk passed with JDK v1.8.0_66
        +1 compile 2m 7s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 29s trunk passed
        +1 mvneclipse 0m 38s trunk passed
        +1 findbugs 3m 30s trunk passed
        +1 javadoc 1m 22s trunk passed with JDK v1.8.0_66
        +1 javadoc 3m 38s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 23s Maven dependency ordering for patch
        +1 mvninstall 1m 12s the patch passed
        +1 compile 1m 45s the patch passed with JDK v1.8.0_66
        +1 javac 1m 45s the patch passed
        +1 compile 2m 4s the patch passed with JDK v1.7.0_91
        +1 javac 2m 4s the patch passed
        -1 checkstyle 0m 31s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237).
        +1 mvnsite 1m 27s the patch passed
        +1 mvneclipse 0m 31s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 findbugs 3m 49s the patch passed
        +1 javadoc 1m 16s the patch passed with JDK v1.8.0_66
        +1 javadoc 3m 32s the patch passed with JDK v1.7.0_91
        +1 unit 0m 20s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 35s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 22s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
        +1 unit 2m 7s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 9m 0s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        64m 25s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782122/YARN-4579.004.patch
        JIRA Issue YARN-4579
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 596cc99a89a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 321072b
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10268/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10268/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10268/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 40s Maven dependency ordering for branch +1 mvninstall 7m 48s trunk passed +1 compile 1m 53s trunk passed with JDK v1.8.0_66 +1 compile 2m 7s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 29s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 3m 30s trunk passed +1 javadoc 1m 22s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 38s trunk passed with JDK v1.7.0_91 0 mvndep 0m 23s Maven dependency ordering for patch +1 mvninstall 1m 12s the patch passed +1 compile 1m 45s the patch passed with JDK v1.8.0_66 +1 javac 1m 45s the patch passed +1 compile 2m 4s the patch passed with JDK v1.7.0_91 +1 javac 2m 4s the patch passed -1 checkstyle 0m 31s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237). +1 mvnsite 1m 27s the patch passed +1 mvneclipse 0m 31s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 3m 49s the patch passed +1 javadoc 1m 16s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 32s the patch passed with JDK v1.7.0_91 +1 unit 0m 20s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 1m 53s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 35s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 0m 22s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 7s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 9m 0s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 64m 25s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782122/YARN-4579.004.patch JIRA Issue YARN-4579 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 596cc99a89a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 321072b Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10268/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10268/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10268/console This message was automatically generated.
        Hide
        rchiang Ray Chiang added a comment -

        RE: checkstyle

        • The YarnConfiguration class was already over the file length limit.
        • The 3 long lines are due to the extra long class name + variable name.
        Show
        rchiang Ray Chiang added a comment - RE: checkstyle The YarnConfiguration class was already over the file length limit. The 3 long lines are due to the extra long class name + variable name.
        Hide
        rkanter Robert Kanter added a comment -

        One minor thing:

        • delCheck isn't actually used anywhere. Given that the test calls deleteTmpFiles() at the end, do we even need to delete here?

        +1 after that.

        Vinod Kumar Vavilapalli, any other comments?

        Show
        rkanter Robert Kanter added a comment - One minor thing: delCheck isn't actually used anywhere. Given that the test calls deleteTmpFiles() at the end, do we even need to delete here? +1 after that. Vinod Kumar Vavilapalli , any other comments?
        Hide
        rchiang Ray Chiang added a comment -

        Yes, the delete needs to occur, since we create the same two log directories through each iteration of the permissionsArray loop and I don't think the eventual mkdir call will work if the directory already exists with different permissions.

        And I'll remove the boolean--I had it there in case I needed to check.

        I found one more bug in the loop so that the permissions would change properly with each iteration of the loop. And I removed the debugging output.

        Show
        rchiang Ray Chiang added a comment - Yes, the delete needs to occur, since we create the same two log directories through each iteration of the permissionsArray loop and I don't think the eventual mkdir call will work if the directory already exists with different permissions. And I'll remove the boolean--I had it there in case I needed to check. I found one more bug in the loop so that the permissions would change properly with each iteration of the loop. And I removed the debugging output.
        Hide
        rchiang Ray Chiang added a comment -
        • Fixed unit test to pass-down permissions properly to DCE
        • Removed debugging output and unused boolean variable
        Show
        rchiang Ray Chiang added a comment - Fixed unit test to pass-down permissions properly to DCE Removed debugging output and unused boolean variable
        Hide
        rkanter Robert Kanter added a comment -

        Makes sense. One final thing:

        • Can you replace for (int i = 0; i < permissionsArray.length; ++i) { with a foreach-style loop?
        Show
        rkanter Robert Kanter added a comment - Makes sense. One final thing: Can you replace for (int i = 0; i < permissionsArray.length; ++i) { with a foreach-style loop?
        Hide
        rchiang Ray Chiang added a comment -

        I will do that.

        Show
        rchiang Ray Chiang added a comment - I will do that.
        Hide
        rchiang Ray Chiang added a comment -

        Fix loop construct

        Show
        rchiang Ray Chiang added a comment - Fix loop construct
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 26s Maven dependency ordering for branch
        +1 mvninstall 7m 41s trunk passed
        +1 compile 1m 51s trunk passed with JDK v1.8.0_66
        +1 compile 2m 10s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 28s trunk passed
        +1 mvneclipse 0m 38s trunk passed
        +1 findbugs 3m 35s trunk passed
        +1 javadoc 1m 24s trunk passed with JDK v1.8.0_66
        +1 javadoc 3m 42s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 25s Maven dependency ordering for patch
        +1 mvninstall 1m 12s the patch passed
        +1 compile 1m 44s the patch passed with JDK v1.8.0_66
        +1 javac 1m 44s the patch passed
        +1 compile 2m 3s the patch passed with JDK v1.7.0_91
        +1 javac 2m 3s the patch passed
        -1 checkstyle 0m 29s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237).
        +1 mvnsite 1m 22s the patch passed
        +1 mvneclipse 0m 32s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 findbugs 3m 55s the patch passed
        +1 javadoc 1m 17s the patch passed with JDK v1.8.0_66
        +1 javadoc 3m 31s the patch passed with JDK v1.7.0_91
        +1 unit 0m 19s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 52s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 29s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 26s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
        +1 unit 2m 8s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 9m 4s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        64m 7s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782149/YARN-4579.005.patch
        JIRA Issue YARN-4579
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux ca1a520cc62c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 9b33a03
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10270/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10270/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10270/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 26s Maven dependency ordering for branch +1 mvninstall 7m 41s trunk passed +1 compile 1m 51s trunk passed with JDK v1.8.0_66 +1 compile 2m 10s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 28s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 3m 35s trunk passed +1 javadoc 1m 24s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 42s trunk passed with JDK v1.7.0_91 0 mvndep 0m 25s Maven dependency ordering for patch +1 mvninstall 1m 12s the patch passed +1 compile 1m 44s the patch passed with JDK v1.8.0_66 +1 javac 1m 44s the patch passed +1 compile 2m 3s the patch passed with JDK v1.7.0_91 +1 javac 2m 3s the patch passed -1 checkstyle 0m 29s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237). +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 32s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 3m 55s the patch passed +1 javadoc 1m 17s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 31s the patch passed with JDK v1.7.0_91 +1 unit 0m 19s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 1m 52s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 29s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 0m 26s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 8s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 9m 4s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 64m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782149/YARN-4579.005.patch JIRA Issue YARN-4579 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ca1a520cc62c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9b33a03 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10270/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10270/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10270/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 26s Maven dependency ordering for branch
        +1 mvninstall 8m 25s trunk passed
        +1 compile 2m 6s trunk passed with JDK v1.8.0_66
        +1 compile 2m 19s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 35s trunk passed
        +1 mvnsite 1m 38s trunk passed
        +1 mvneclipse 0m 38s trunk passed
        +1 findbugs 3m 53s trunk passed
        +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66
        +1 javadoc 3m 52s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 25s Maven dependency ordering for patch
        +1 mvninstall 1m 20s the patch passed
        +1 compile 1m 59s the patch passed with JDK v1.8.0_66
        +1 javac 1m 59s the patch passed
        +1 compile 2m 17s the patch passed with JDK v1.7.0_91
        +1 javac 2m 17s the patch passed
        -1 checkstyle 0m 31s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237).
        +1 mvnsite 1m 30s the patch passed
        +1 mvneclipse 0m 33s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 4m 9s the patch passed
        +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66
        +1 javadoc 3m 48s the patch passed with JDK v1.7.0_91
        +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
        +1 unit 2m 0s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 46s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
        +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 9m 16s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        68m 33s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782151/YARN-4579.006.patch
        JIRA Issue YARN-4579
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 9d37bd081b32 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / fb64e60
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10271/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10271/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10271/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 26s Maven dependency ordering for branch +1 mvninstall 8m 25s trunk passed +1 compile 2m 6s trunk passed with JDK v1.8.0_66 +1 compile 2m 19s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 35s trunk passed +1 mvnsite 1m 38s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 3m 53s trunk passed +1 javadoc 1m 35s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 52s trunk passed with JDK v1.7.0_91 0 mvndep 0m 25s Maven dependency ordering for patch +1 mvninstall 1m 20s the patch passed +1 compile 1m 59s the patch passed with JDK v1.8.0_66 +1 javac 1m 59s the patch passed +1 compile 2m 17s the patch passed with JDK v1.7.0_91 +1 javac 2m 17s the patch passed -1 checkstyle 0m 31s Patch generated 4 new checkstyle issues in hadoop-yarn-project/hadoop-yarn (total was 234, now 237). +1 mvnsite 1m 30s the patch passed +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 4m 9s the patch passed +1 javadoc 1m 25s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 48s the patch passed with JDK v1.7.0_91 +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 2m 0s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 46s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 9m 16s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 68m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782151/YARN-4579.006.patch JIRA Issue YARN-4579 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 9d37bd081b32 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fb64e60 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/10271/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10271/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10271/console This message was automatically generated.
        Hide
        rchiang Ray Chiang added a comment -

        RE: Checkstyle

        Same issues as before.

        Show
        rchiang Ray Chiang added a comment - RE: Checkstyle Same issues as before.
        Hide
        rkanter Robert Kanter added a comment -

        +1

        Show
        rkanter Robert Kanter added a comment - +1
        Hide
        rkanter Robert Kanter added a comment -

        Thanks Ray. Committed to trunk and branch-2!

        Show
        rkanter Robert Kanter added a comment - Thanks Ray. Committed to trunk and branch-2!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9372 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9372/)
        YARN-4579. Allow DefaultContainerExecutor container log directory (rkanter: rev d7fdec1e6b465395d2faca6e404e329d20f6c3d8)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9372 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9372/ ) YARN-4579 . Allow DefaultContainerExecutor container log directory (rkanter: rev d7fdec1e6b465395d2faca6e404e329d20f6c3d8) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml hadoop-yarn-project/CHANGES.txt
        Hide
        rchiang Ray Chiang added a comment -

        Thanks for the review and the commit!

        Show
        rchiang Ray Chiang added a comment - Thanks for the review and the commit!

          People

          • Assignee:
            rchiang Ray Chiang
            Reporter:
            rchiang Ray Chiang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development