Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5287

LinuxContainerExecutor fails to set proper permission

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.7.2
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: nodemanager
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      LinuxContainerExecutor fails to set the proper permissions on the local directories(i.e., /hadoop/yarn/local/usercache/... by default) if the cluster has been configured with a restrictive umask, e.g.: umask 077. Job failed due to the following reason:
      Path /hadoop/yarn/local/usercache/ambari-qa/appcache/application_xxxxxxxx has permission 700 but needs permission 750

      1. YARN-5287.003.patch
        5 kB
        Ying Zhang
      2. YARN-5287.004.patch
        6 kB
        Ying Zhang
      3. YARN-5287.005.patch
        6 kB
        Ying Zhang
      4. YARN-5287.branch-2.001.patch
        6 kB
        Naganarasimha G R
      5. YARN-5287-tmp.patch
        0.8 kB
        Naganarasimha G R

        Issue Links

          Activity

          Hide
          Ying Zhang Ying Zhang added a comment - - edited

          The fix would be explicitly setting the permission after creating the directories. Similar as what has been done in YARN-88 for DefaultContainerExecutor.

          Show
          Ying Zhang Ying Zhang added a comment - - edited The fix would be explicitly setting the permission after creating the directories. Similar as what has been done in YARN-88 for DefaultContainerExecutor.
          Hide
          Ying Zhang Ying Zhang added a comment -

          Way to reproduce:
          1. configure the cluster with umask 077 for all nodes
          (for example, modify the /etc/profile and /etc/bashrc)
          2. enable LinuxContainerExecutor through Ambari or config file. Would also need to set "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" to false under non-secure mode.
          3. restart all affected
          4. run a simple MR job

          Show
          Ying Zhang Ying Zhang added a comment - Way to reproduce: 1. configure the cluster with umask 077 for all nodes (for example, modify the /etc/profile and /etc/bashrc) 2. enable LinuxContainerExecutor through Ambari or config file. Would also need to set "yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users" to false under non-secure mode. 3. restart all affected 4. run a simple MR job
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 9m 54s trunk passed
          +1 compile 0m 32s trunk passed
          +1 mvnsite 0m 34s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 mvninstall 0m 28s the patch passed
          +1 compile 0m 35s the patch passed
          +1 cc 0m 35s the patch passed
          +1 javac 0m 35s the patch passed
          +1 mvnsite 0m 32s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 13m 54s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          27m 52s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817322/YARN-5287.001.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux b06dc3eab25c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f292624
          Default Java 1.8.0_91
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12281/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12281/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 54s trunk passed +1 compile 0m 32s trunk passed +1 mvnsite 0m 34s trunk passed +1 mvneclipse 0m 13s trunk passed +1 mvninstall 0m 28s the patch passed +1 compile 0m 35s the patch passed +1 cc 0m 35s the patch passed +1 javac 0m 35s the patch passed +1 mvnsite 0m 32s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 13m 54s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 27m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817322/YARN-5287.001.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux b06dc3eab25c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f292624 Default Java 1.8.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12281/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12281/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          We had also observed a similar issue but the fix we had was little different, let me evaluate your approach to ours.. Any way have assiged the jira to your name !

          Show
          Naganarasimha Naganarasimha G R added a comment - We had also observed a similar issue but the fix we had was little different, let me evaluate your approach to ours.. Any way have assiged the jira to your name !
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Ying Zhang, This was the approach which we had taken to solve the same issue but seems like it would be better to solve the issue based on the approach taken by you as it will get executed only in the failure scenario.
          Can you check any test case can be added for the same ?

          Show
          Naganarasimha Naganarasimha G R added a comment - Ying Zhang , This was the approach which we had taken to solve the same issue but seems like it would be better to solve the issue based on the approach taken by you as it will get executed only in the failure scenario. Can you check any test case can be added for the same ?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          further create_directory_for_user also uses the same approach...

          Show
          Naganarasimha Naganarasimha G R added a comment - further create_directory_for_user also uses the same approach...
          Hide
          Ying Zhang Ying Zhang added a comment -

          Thanks Naganarasimha. I've tested my change with 3-node cluster, and it works as expected.
          I've currently working on adding unit test, and I've found the current test-container-executor is not working when running as root (which is required for testing initialize app or run container), for example, under root, run "test-container-executor <myusername>". So I've made some minor change there. Have you encountered the same problem?

          Show
          Ying Zhang Ying Zhang added a comment - Thanks Naganarasimha. I've tested my change with 3-node cluster, and it works as expected. I've currently working on adding unit test, and I've found the current test-container-executor is not working when running as root (which is required for testing initialize app or run container), for example, under root, run "test-container-executor <myusername>". So I've made some minor change there. Have you encountered the same problem?
          Hide
          Ying Zhang Ying Zhang added a comment -

          sure, working on the test case right now.

          Show
          Ying Zhang Ying Zhang added a comment - sure, working on the test case right now.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          We faced issue when umask was set to 077. Well basically issue was identified and fixed by Surendra Singh Lilhore. May be he can add more light on the test failure if any...

          Show
          Naganarasimha Naganarasimha G R added a comment - We faced issue when umask was set to 077. Well basically issue was identified and fixed by Surendra Singh Lilhore . May be he can add more light on the test failure if any...
          Hide
          Ying Zhang Ying Zhang added a comment -

          Hi Naganarasimha, I've uploaded a new patch with a new test case included. I've run the "test-container-executor" with regular user and root. It passes.
          As I said earlier, there is issue with the current test-container-executor when running under root. I have made some minor change to make it pass(to be specific, in test-container-executor.c, in function main(), when running as root, test_recursive_unlink_children() needs to be run before set_user(username). Not sure if it is correct, just a work-around). I don't think I should include it in this patch. Let me know your idea. Thank you.

          Show
          Ying Zhang Ying Zhang added a comment - Hi Naganarasimha, I've uploaded a new patch with a new test case included. I've run the "test-container-executor" with regular user and root. It passes. As I said earlier, there is issue with the current test-container-executor when running under root. I have made some minor change to make it pass(to be specific, in test-container-executor.c, in function main(), when running as root, test_recursive_unlink_children() needs to be run before set_user(username). Not sure if it is correct, just a work-around). I don't think I should include it in this patch. Let me know your idea. Thank you.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 32s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 26s trunk passed
          +1 compile 0m 28s trunk passed
          +1 mvnsite 0m 31s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 mvninstall 0m 28s the patch passed
          +1 compile 0m 25s the patch passed
          -1 cc 0m 25s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3)
          +1 javac 0m 25s the patch passed
          +1 mvnsite 0m 31s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 unit 13m 38s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          25m 0s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817875/YARN-5287.002.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux c46f308452c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dbe97aa
          Default Java 1.8.0_91
          cc https://builds.apache.org/job/PreCommit-YARN-Build/12323/artifact/patchprocess/diff-compile-cc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12323/artifact/patchprocess/whitespace-eol.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12323/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12323/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 32s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 26s trunk passed +1 compile 0m 28s trunk passed +1 mvnsite 0m 31s trunk passed +1 mvneclipse 0m 15s trunk passed +1 mvninstall 0m 28s the patch passed +1 compile 0m 25s the patch passed -1 cc 0m 25s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3) +1 javac 0m 25s the patch passed +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 10s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 unit 13m 38s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 25m 0s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817875/YARN-5287.002.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux c46f308452c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dbe97aa Default Java 1.8.0_91 cc https://builds.apache.org/job/PreCommit-YARN-Build/12323/artifact/patchprocess/diff-compile-cc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/12323/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12323/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12323/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Modified attached file name...

          Show
          Naganarasimha Naganarasimha G R added a comment - Modified attached file name...
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 34s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 8m 18s trunk passed
          +1 compile 0m 35s trunk passed
          +1 mvnsite 0m 32s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 mvninstall 0m 32s the patch passed
          +1 compile 0m 32s the patch passed
          +1 cc 0m 32s the patch passed
          +1 javac 0m 32s the patch passed
          +1 mvnsite 0m 31s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 13m 40s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          26m 26s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817913/YARN-5287-tmp.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux 92382ca50266 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / dbe97aa
          Default Java 1.8.0_91
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12324/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12324/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 34s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 18s trunk passed +1 compile 0m 35s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 16s trunk passed +1 mvninstall 0m 32s the patch passed +1 compile 0m 32s the patch passed +1 cc 0m 32s the patch passed +1 javac 0m 32s the patch passed +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 13m 40s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 26m 26s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817913/YARN-5287-tmp.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux 92382ca50266 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / dbe97aa Default Java 1.8.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12324/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12324/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 21s trunk passed
          +1 compile 0m 27s trunk passed
          +1 mvnsite 0m 27s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 22s the patch passed
          +1 cc 0m 22s the patch passed
          +1 javac 0m 22s the patch passed
          +1 mvnsite 0m 25s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 13m 5s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          23m 42s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817919/YARN-5287.003.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux 3104045f2e36 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / be26c1b
          Default Java 1.8.0_91
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12325/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12325/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 21s trunk passed +1 compile 0m 27s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 13s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 22s the patch passed +1 cc 0m 22s the patch passed +1 javac 0m 22s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 13m 5s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 23m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12817919/YARN-5287.003.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux 3104045f2e36 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / be26c1b Default Java 1.8.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12325/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12325/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Hi Ying Zhang,
          Please dont delete the previous artifacts it would be helpful for tracking.

          Show
          Naganarasimha Naganarasimha G R added a comment - Hi Ying Zhang , Please dont delete the previous artifacts it would be helpful for tracking.
          Hide
          Ying Zhang Ying Zhang added a comment - - edited

          My fault. Got it.

          Show
          Ying Zhang Ying Zhang added a comment - - edited My fault. Got it.
          Hide
          Ying Zhang Ying Zhang added a comment -

          Hi Naganarasimha,
          Would you please help to review the patch when you're available? Thanks.

          Show
          Ying Zhang Ying Zhang added a comment - Hi Naganarasimha, Would you please help to review the patch when you're available? Thanks.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Ying Zhang, sorry for the long delay. I compiled and tested your patch it seems fine to me, will commit it shortly if no one has any concerns on this patch !

          Show
          Naganarasimha Naganarasimha G R added a comment - Ying Zhang , sorry for the long delay. I compiled and tested your patch it seems fine to me, will commit it shortly if no one has any concerns on this patch !
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          Thanks Ying Zhang for the patch!! I have been encountered such issues when I set default umask other than 0022.

          Regarding the patch, one comment is chmod(npath, perm) should be covered with error handling. Otherwise, any failure case for chmod will be continued which will cause problem later.

          Show
          rohithsharma Rohith Sharma K S added a comment - Thanks Ying Zhang for the patch!! I have been encountered such issues when I set default umask other than 0022. Regarding the patch, one comment is chmod(npath, perm) should be covered with error handling. Otherwise, any failure case for chmod will be continued which will cause problem later.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 8s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 9s trunk passed
          +1 compile 0m 28s trunk passed
          +1 mvnsite 0m 28s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 mvninstall 0m 23s the patch passed
          +1 compile 0m 25s the patch passed
          +1 cc 0m 25s the patch passed
          +1 javac 0m 25s the patch passed
          +1 mvnsite 0m 26s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 unit 13m 6s hadoop-yarn-server-nodemanager in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          42m 28s



          Reason Tests
          Failed junit tests hadoop.yarn.server.nodemanager.TestDirectoryCollection



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820174/YARN-5287.004.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux 6693c1fa18bf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f1a4863
          Default Java 1.8.0_101
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12504/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12504/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12504/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12504/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 9s trunk passed +1 compile 0m 28s trunk passed +1 mvnsite 0m 28s trunk passed +1 mvneclipse 0m 13s trunk passed +1 mvninstall 0m 23s the patch passed +1 compile 0m 25s the patch passed +1 cc 0m 25s the patch passed +1 javac 0m 25s the patch passed +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 unit 13m 6s hadoop-yarn-server-nodemanager in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 42m 28s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.TestDirectoryCollection Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820174/YARN-5287.004.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux 6693c1fa18bf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f1a4863 Default Java 1.8.0_101 unit https://builds.apache.org/job/PreCommit-YARN-Build/12504/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12504/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12504/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12504/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Ying Zhang Ying Zhang added a comment -

          Thanks Naganarasimha and Rohith for review. I've updated the patch with addressing Rohith's comment. Please have a look.
          The test failure is not related, separate code path. And I've seen YARN-5425 has already been filed against it. Although what's interesting is that the test failure seems to be similar with what we are trying to resolve here.

          Show
          Ying Zhang Ying Zhang added a comment - Thanks Naganarasimha and Rohith for review. I've updated the patch with addressing Rohith's comment. Please have a look. The test failure is not related, separate code path. And I've seen YARN-5425 has already been filed against it. Although what's interesting is that the test failure seems to be similar with what we are trying to resolve here.
          Hide
          Ying Zhang Ying Zhang added a comment -

          Naganarasimha G R and Rohith Sharma K S, let me know your comments

          Show
          Ying Zhang Ying Zhang added a comment - Naganarasimha G R and Rohith Sharma K S , let me know your comments
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 39s trunk passed
          +1 compile 0m 28s trunk passed
          +1 mvnsite 0m 32s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 mvninstall 0m 27s the patch passed
          +1 compile 0m 29s the patch passed
          -1 cc 0m 29s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 1 new + 2 unchanged - 1 fixed = 3 total (was 3)
          +1 javac 0m 29s the patch passed
          +1 mvnsite 0m 29s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 unit 13m 18s hadoop-yarn-server-nodemanager in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          24m 34s



          Reason Tests
          Failed junit tests hadoop.yarn.server.nodemanager.TestDirectoryCollection



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820622/YARN-5287.004.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux af775f3278d7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8d06bda
          Default Java 1.8.0_101
          cc https://builds.apache.org/job/PreCommit-YARN-Build/12533/artifact/patchprocess/diff-compile-cc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/12533/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12533/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12533/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12533/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 39s trunk passed +1 compile 0m 28s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 15s trunk passed +1 mvninstall 0m 27s the patch passed +1 compile 0m 29s the patch passed -1 cc 0m 29s hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 1 new + 2 unchanged - 1 fixed = 3 total (was 3) +1 javac 0m 29s the patch passed +1 mvnsite 0m 29s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 unit 13m 18s hadoop-yarn-server-nodemanager in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 24m 34s Reason Tests Failed junit tests hadoop.yarn.server.nodemanager.TestDirectoryCollection Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820622/YARN-5287.004.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux af775f3278d7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8d06bda Default Java 1.8.0_101 cc https://builds.apache.org/job/PreCommit-YARN-Build/12533/artifact/patchprocess/diff-compile-cc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/12533/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/12533/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12533/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12533/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          The current change look good to me. And as I am not very much comfortable in C code, I would like to hear other folks opinion also.
          Xuan Gong Would you help us to review this patch please?

          Show
          rohithsharma Rohith Sharma K S added a comment - The current change look good to me. And as I am not very much comfortable in C code, I would like to hear other folks opinion also. Xuan Gong Would you help us to review this patch please?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks for the patch Ying Zhang,
          +1 , Latest patch LGTM, If no more comments will wait for some time and commit it !

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks for the patch Ying Zhang , +1 , Latest patch LGTM, If no more comments will wait for some time and commit it !
          Hide
          vvasudev Varun Vasudev added a comment -

          Ying Zhang the patch no longer applies cleanly on trunk.

          It would be great if you could make a few minor changes -
          1)

          +/**
          + *  * Function to prepare the container directories.
          + *   * It creates the container work and log directories.
          + *    */
          

          Please change the comment to follow the same format as other comments(no need for the extra "*" on the individual lines)

          2)

          +// This test is used to verify that app and container directories can be
          +// created with required permissions when umask has been set to a restrictive
          +// value of 077.
          

          Change the formatting to follow

          /** .. */

          3)

          +  //Create container directories for "app_5"

          Add space between // and Create

          Show
          vvasudev Varun Vasudev added a comment - Ying Zhang the patch no longer applies cleanly on trunk. It would be great if you could make a few minor changes - 1) +/** + * * Function to prepare the container directories. + * * It creates the container work and log directories. + * */ Please change the comment to follow the same format as other comments(no need for the extra "*" on the individual lines) 2) + // This test is used to verify that app and container directories can be + // created with required permissions when umask has been set to a restrictive + // value of 077. Change the formatting to follow /** .. */ 3) + //Create container directories for "app_5" Add space between // and Create
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 2s trunk passed
          +1 compile 0m 28s trunk passed
          +1 mvnsite 0m 27s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 mvninstall 0m 22s the patch passed
          +1 compile 0m 24s the patch passed
          +1 cc 0m 24s the patch passed
          +1 javac 0m 24s the patch passed
          +1 mvnsite 0m 24s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 13m 8s hadoop-yarn-server-nodemanager in the patch passed.
          +1 asflicense 0m 15s The patch does not generate ASF License warnings.
          23m 27s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821811/YARN-5287.005.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux 11e214400a4a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d848184
          Default Java 1.8.0_101
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12625/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12625/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 2s trunk passed +1 compile 0m 28s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 13s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 24s the patch passed +1 cc 0m 24s the patch passed +1 javac 0m 24s the patch passed +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 13m 8s hadoop-yarn-server-nodemanager in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 23m 27s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821811/YARN-5287.005.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux 11e214400a4a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d848184 Default Java 1.8.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12625/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12625/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Ying Zhang Ying Zhang added a comment -

          Thanks Naganarasimha G R and Varun Vasudev. Comments addressed and patch updated. Please have a look and kindly help to commit it.

          Show
          Ying Zhang Ying Zhang added a comment - Thanks Naganarasimha G R and Varun Vasudev . Comments addressed and patch updated. Please have a look and kindly help to commit it.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          I verified the latest patch and seems to rectify all the comments, so hope i did not miss any nits in the patch, Varun Vasudev & Rohith Sharma K S anything else needs to be addressed?

          Show
          Naganarasimha Naganarasimha G R added a comment - I verified the latest patch and seems to rectify all the comments, so hope i did not miss any nits in the patch, Varun Vasudev & Rohith Sharma K S anything else needs to be addressed?
          Hide
          vvasudev Varun Vasudev added a comment -

          Looks ok to me.

          Show
          vvasudev Varun Vasudev added a comment - Looks ok to me.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks for the review Varun Vasudev, Will commit this patch shortly !

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks for the review Varun Vasudev , Will commit this patch shortly !
          Hide
          rohithsharma Rohith Sharma K S added a comment -

          +1 LGTM

          Show
          rohithsharma Rohith Sharma K S added a comment - +1 LGTM
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10230 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10230/)
          YARN-5287. LinuxContainerExecutor fails to set proper permission. (naganarasimha_gr: rev 131d58a24edcf3b492a7dd0fa5bb3dbf27daf95d)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10230 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10230/ ) YARN-5287 . LinuxContainerExecutor fails to set proper permission. (naganarasimha_gr: rev 131d58a24edcf3b492a7dd0fa5bb3dbf27daf95d) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c
          Hide
          Naganarasimha Naganarasimha G R added a comment - - edited

          Thanks for the contributions Ying Zhang and review by Rohith Sharma K S & Varun Vasudev, Seems like patch fails to apply on Branch-2, currently working on re basing the patch for branch-2. Was also planning to commit this in 2.8 too ...

          Show
          Naganarasimha Naganarasimha G R added a comment - - edited Thanks for the contributions Ying Zhang and review by Rohith Sharma K S & Varun Vasudev , Seems like patch fails to apply on Branch-2, currently working on re basing the patch for branch-2. Was also planning to commit this in 2.8 too ...
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          hi Ying Zhang,
          Can you review this patch for branch-2 ?

          Show
          Naganarasimha Naganarasimha G R added a comment - hi Ying Zhang , Can you review this patch for branch-2 ?
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 17m 42s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 47s branch-2 passed
          +1 compile 0m 24s branch-2 passed with JDK v1.8.0_101
          +1 compile 0m 28s branch-2 passed with JDK v1.7.0_101
          +1 mvnsite 0m 31s branch-2 passed
          +1 mvneclipse 0m 14s branch-2 passed
          +1 mvninstall 0m 25s the patch passed
          +1 compile 0m 21s the patch passed with JDK v1.8.0_101
          +1 cc 0m 21s the patch passed
          +1 javac 0m 21s the patch passed
          +1 compile 0m 25s the patch passed with JDK v1.7.0_101
          +1 cc 0m 25s the patch passed
          +1 javac 0m 25s the patch passed
          +1 mvnsite 0m 27s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 12m 58s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_101.
          +1 unit 13m 46s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_101.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          55m 19s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:b59b8b7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822477/YARN-5287.branch-2.001.patch
          JIRA Issue YARN-5287
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux 88905281c52d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 9cdd97a
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12664/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/12664/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 17m 42s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 47s branch-2 passed +1 compile 0m 24s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 28s branch-2 passed with JDK v1.7.0_101 +1 mvnsite 0m 31s branch-2 passed +1 mvneclipse 0m 14s branch-2 passed +1 mvninstall 0m 25s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_101 +1 cc 0m 21s the patch passed +1 javac 0m 21s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_101 +1 cc 0m 25s the patch passed +1 javac 0m 25s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 12m 58s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_101. +1 unit 13m 46s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_101. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 55m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822477/YARN-5287.branch-2.001.patch JIRA Issue YARN-5287 Optional Tests asflicense compile cc mvnsite javac unit uname Linux 88905281c52d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 9cdd97a Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12664/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/12664/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Ying Zhang Ying Zhang added a comment -

          Thanks Naganarasimha G R. Looks good.

          Show
          Ying Zhang Ying Zhang added a comment - Thanks Naganarasimha G R . Looks good.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Committed it to 2.8 and 2.9 branches too..

          Show
          Naganarasimha Naganarasimha G R added a comment - Committed it to 2.8 and 2.9 branches too..
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Reopening the issue due to blocker issue(YARN-5765) introduced by the patch. reverted the commits in branch-2.8 and branch-2 but revert fails in trunk, working on it...

          Show
          Naganarasimha Naganarasimha G R added a comment - Reopening the issue due to blocker issue( YARN-5765 ) introduced by the patch. reverted the commits in branch-2.8 and branch-2 but revert fails in trunk, working on it...
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          As this jira is committed to 3.0.0-alpha1 and release is given.
          Hence, let's please do so under a new JIRA since this was released in 3.0.0-alpha1.
          This way the change-log for alpha1 -> alpha2 will show the revert.

          Show
          brahmareddy Brahma Reddy Battula added a comment - As this jira is committed to 3.0.0-alpha1 and release is given. Hence, let's please do so under a new JIRA since this was released in 3.0.0-alpha1. This way the change-log for alpha1 -> alpha2 will show the revert.
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks Brahma, But this part of it i am already discussing in YARN-5765...

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks Brahma , But this part of it i am already discussing in YARN-5765 ...
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Reverted in the trunk too

          Show
          Naganarasimha Naganarasimha G R added a comment - Reverted in the trunk too
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10836 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10836/)
          Reverted due to issue YARN-5765. Revert "YARN-5287. (naganarasimha_gr: rev 43aef303bf9b71293b00c7ed6e8807d15274ca95)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10836 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10836/ ) Reverted due to issue YARN-5765 . Revert " YARN-5287 . (naganarasimha_gr: rev 43aef303bf9b71293b00c7ed6e8807d15274ca95) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.h (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          Hide
          brahmareddy Brahma Reddy Battula added a comment - - edited

          AFAIK,As alpha1 is already released with this jira,Ideally we should use different jira to revert this( which might leads change log conflict b/w alpha1 and aplha2)..

          Andrew Wang can you please correct me if I am wrong here..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - - edited AFAIK,As alpha1 is already released with this jira,Ideally we should use different jira to revert this( which might leads change log conflict b/w alpha1 and aplha2).. Andrew Wang can you please correct me if I am wrong here..?
          Hide
          varun_saxena Varun Saxena added a comment -

          Brahma Reddy Battula IIUC, we now use individual JIRAs' to build release notes.
          YARN-5765 is already there which can be used to track the revert. Which is similar to what has been done for YARN-5882 (which will be used to track revert of YARN-4126).
          While reverting YARN-5287, YARN-5765 has been mentioned in the commit message. The only thing which needs to be done is close that JIRA and mark fix version as 3.0.0-alpha2. Shouldn't that be enough ? Thoughts ?

          Show
          varun_saxena Varun Saxena added a comment - Brahma Reddy Battula IIUC, we now use individual JIRAs' to build release notes. YARN-5765 is already there which can be used to track the revert. Which is similar to what has been done for YARN-5882 (which will be used to track revert of YARN-4126 ). While reverting YARN-5287 , YARN-5765 has been mentioned in the commit message. The only thing which needs to be done is close that JIRA and mark fix version as 3.0.0-alpha2. Shouldn't that be enough ? Thoughts ?
          Hide
          Naganarasimha Naganarasimha G R added a comment -

          Thanks for sharing the thoughts Brahma and Varun Saxena, i was planning to do the same and also had mentioned the same as a comment in YARN-5765. Along with that would be rewording the defect summary and description. Please inform anything else also needs to be taken care.

          Show
          Naganarasimha Naganarasimha G R added a comment - Thanks for sharing the thoughts Brahma and Varun Saxena , i was planning to do the same and also had mentioned the same as a comment in YARN-5765 . Along with that would be rewording the defect summary and description. Please inform anything else also needs to be taken care.
          Hide
          andrew.wang Andrew Wang added a comment -

          Thanks for the ping Brahma, seems like we're all on the same page here. As long as the revert JIRA summary mentions the JIRA being reverted, it should be clear to end-users.

          Show
          andrew.wang Andrew Wang added a comment - Thanks for the ping Brahma, seems like we're all on the same page here. As long as the revert JIRA summary mentions the JIRA being reverted, it should be clear to end-users.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Removed 2.8.0 from fix version because it was reverted before 2.8.0 release.

          Show
          ajisakaa Akira Ajisaka added a comment - Removed 2.8.0 from fix version because it was reverted before 2.8.0 release.
          Hide
          jianhe Jian He added a comment -

          Naganarasimha G R, Haibo Chen, saw a bunch of comments on YARN-5765. As the patch is reverted,
          Is there any plan to fix this original issue ?

          Show
          jianhe Jian He added a comment - Naganarasimha G R , Haibo Chen , saw a bunch of comments on YARN-5765 . As the patch is reverted, Is there any plan to fix this original issue ?

            People

            • Assignee:
              Naganarasimha Naganarasimha G R
              Reporter:
              Ying Zhang Ying Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 48h
                48h
                Remaining:
                Remaining Estimate - 48h
                48h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development