Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6150

TestContainerManagerSecurity tests for Yarn Server are flakey

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-beta1
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Repeated runs of org.apache.hadoop.yarn.server.TestContainerManagedSecurity can either pass or fail on repeated runs on the same codebase. Also, the two runs (one in secure mode, one without security) aren't well labeled in JUnit.

      1. YARN-6150.001.patch
        4 kB
        Daniel Sturman
      2. YARN-6150.002.patch
        4 kB
        Daniel Sturman
      3. YARN-6150.003.patch
        4 kB
        Daniel Sturman
      4. YARN-6150.004.patch
        5 kB
        Daniel Sturman
      5. YARN-6150.005.patch
        9 kB
        Daniel Sturman
      6. YARN-6150.006.patch
        9 kB
        Daniel Sturman
      7. YARN-6150.007.patch
        10 kB
        Ray Chiang

        Issue Links

          Activity

          Hide
          sturman Daniel Sturman added a comment -

          The issue in the test was that a test assumed it would see Containers COMPLETE before being removed. The asynchronous nature of the test meant in some cases Containers could COMPLETE and then be removed before the test waiting to see complete would see them (it would be sleeping). This resulted in a NullPointerException when it tried to look up relevant Container. Since this was timing based it didn't occur each time.

          The fix is to just not do that lookup each time through the while loop, as the Container object isn't being deleted but rather just moved.

          Show
          sturman Daniel Sturman added a comment - The issue in the test was that a test assumed it would see Containers COMPLETE before being removed. The asynchronous nature of the test meant in some cases Containers could COMPLETE and then be removed before the test waiting to see complete would see them (it would be sleeping). This resulted in a NullPointerException when it tried to look up relevant Container. Since this was timing based it didn't occur each time. The fix is to just not do that lookup each time through the while loop, as the Container object isn't being deleted but rather just moved.
          Hide
          sturman Daniel Sturman added a comment -

          I also cleaned up the use of JUnit to assign names to the two runs of the test. The first I called Simple as it was run without security. The second I named Secure.

          Show
          sturman Daniel Sturman added a comment - I also cleaned up the use of JUnit to assign names to the two runs of the test. The first I called Simple as it was run without security. The second I named Secure.
          Hide
          templedf Daniel Templeton added a comment - - edited

          Looks generally good to me. One issue I see is that I think you inverted the check for complete on the container--you're now looping while it's complete instead of until it's complete.

          Show
          templedf Daniel Templeton added a comment - - edited Looks generally good to me. One issue I see is that I think you inverted the check for complete on the container--you're now looping while it's complete instead of until it's complete.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 14m 26s trunk passed
          +1 compile 0m 17s trunk passed
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 18s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 11s trunk passed
          +1 mvninstall 0m 15s the patch passed
          +1 compile 0m 13s the patch passed
          +1 javac 0m 13s the patch passed
          -0 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 4 new + 35 unchanged - 3 fixed = 39 total (was 38)
          +1 mvnsite 0m 17s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 8s the patch passed
          -1 unit 6m 38s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          25m 21s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851196/YARN-6150.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 39f7421f8644 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 663e683
          Default Java 1.8.0_121
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14829/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/14829/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14829/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14829/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 26s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 18s trunk passed +1 mvneclipse 0m 15s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 11s trunk passed +1 mvninstall 0m 15s the patch passed +1 compile 0m 13s the patch passed +1 javac 0m 13s the patch passed -0 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 4 new + 35 unchanged - 3 fixed = 39 total (was 38) +1 mvnsite 0m 17s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 8s the patch passed -1 unit 6m 38s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 25m 21s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851196/YARN-6150.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 39f7421f8644 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 663e683 Default Java 1.8.0_121 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14829/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14829/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14829/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/14829/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          sturman Daniel Sturman added a comment -

          Fixed a bug and some style issues in the patch. Will resubmit.

          Show
          sturman Daniel Sturman added a comment - Fixed a bug and some style issues in the patch. Will resubmit.
          Hide
          sturman Daniel Sturman added a comment -

          Had a condition inverted in the test loop. Fixed some style issues that were flagged.

          Show
          sturman Daniel Sturman added a comment - Had a condition inverted in the test loop. Fixed some style issues that were flagged.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 14m 34s trunk passed
          +1 compile 0m 17s trunk passed
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 0m 19s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 10s trunk passed
          +1 mvninstall 0m 19s the patch passed
          +1 compile 0m 17s the patch passed
          +1 javac 0m 17s the patch passed
          +1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 0 new + 35 unchanged - 3 fixed = 35 total (was 38)
          +1 mvnsite 0m 18s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 8s the patch passed
          -1 unit 6m 39s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          25m 36s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851199/YARN-6150.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2b116362f72b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 663e683
          Default Java 1.8.0_121
          unit https://builds.apache.org/job/PreCommit-YARN-Build/14830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14830/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14830/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 34s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 19s trunk passed +1 mvneclipse 0m 16s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 10s trunk passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 17s the patch passed +1 javac 0m 17s the patch passed +1 checkstyle 0m 11s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 0 new + 35 unchanged - 3 fixed = 35 total (was 38) +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 8s the patch passed -1 unit 6m 39s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 25m 36s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851199/YARN-6150.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2b116362f72b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 663e683 Default Java 1.8.0_121 unit https://builds.apache.org/job/PreCommit-YARN-Build/14830/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14830/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/14830/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          sturman Daniel Sturman added a comment -

          hadoop.yarn.server.TestContainerManagerSecurity is failing in the automated unit tests. Can't tell yet if thats a timing issue or something else. Investigating...

          Show
          sturman Daniel Sturman added a comment - hadoop.yarn.server.TestContainerManagerSecurity is failing in the automated unit tests. Can't tell yet if thats a timing issue or something else. Investigating...
          Hide
          sturman Daniel Sturman added a comment -

          Remove the 120s timeout on TestContainerManagedSecurity tests as the upstream check was timing out. My understanding is these sorts of timeouts are no longer desirable.

          Show
          sturman Daniel Sturman added a comment - Remove the 120s timeout on TestContainerManagedSecurity tests as the upstream check was timing out. My understanding is these sorts of timeouts are no longer desirable.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 13m 59s trunk passed
          +1 compile 0m 17s trunk passed
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 18s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 11s trunk passed
          +1 mvninstall 0m 15s the patch passed
          +1 compile 0m 13s the patch passed
          +1 javac 0m 13s the patch passed
          +1 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 0 new + 35 unchanged - 3 fixed = 35 total (was 38)
          +1 mvnsite 0m 16s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 8s the patch passed
          -1 unit 8m 34s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          26m 41s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851245/YARN-6150.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ddf7cebfd804 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d88497d
          Default Java 1.8.0_121
          unit https://builds.apache.org/job/PreCommit-YARN-Build/14832/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14832/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14832/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 59s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 18s trunk passed +1 mvneclipse 0m 15s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 11s trunk passed +1 mvninstall 0m 15s the patch passed +1 compile 0m 13s the patch passed +1 javac 0m 13s the patch passed +1 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 0 new + 35 unchanged - 3 fixed = 35 total (was 38) +1 mvnsite 0m 16s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 8s the patch passed -1 unit 8m 34s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 26m 41s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851245/YARN-6150.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ddf7cebfd804 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d88497d Default Java 1.8.0_121 unit https://builds.apache.org/job/PreCommit-YARN-Build/14832/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14832/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/14832/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          sturman Daniel Sturman added a comment -

          Its annoying that I have to do this but... there's an assertion failure when the Apache infrastructure runs this test but cannot reproduce, so added more information to get some clue as to what is going on.

          Show
          sturman Daniel Sturman added a comment - Its annoying that I have to do this but... there's an assertion failure when the Apache infrastructure runs this test but cannot reproduce, so added more information to get some clue as to what is going on.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 16m 9s trunk passed
          +1 compile 0m 21s trunk passed
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 0m 20s trunk passed
          +1 mvneclipse 0m 17s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 11s trunk passed
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 18s the patch passed
          +1 javac 0m 18s the patch passed
          -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 1 new + 34 unchanged - 4 fixed = 35 total (was 38)
          +1 mvnsite 0m 19s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 8s the patch passed
          -1 unit 8m 47s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          29m 37s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851310/YARN-6150.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ccf4e2ef654b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d88497d
          Default Java 1.8.0_121
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14839/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/14839/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14839/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14839/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 16m 9s trunk passed +1 compile 0m 21s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 20s trunk passed +1 mvneclipse 0m 17s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 11s trunk passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 18s the patch passed +1 javac 0m 18s the patch passed -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 1 new + 34 unchanged - 4 fixed = 35 total (was 38) +1 mvnsite 0m 19s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 8s the patch passed -1 unit 8m 47s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 29m 37s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851310/YARN-6150.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ccf4e2ef654b 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d88497d Default Java 1.8.0_121 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14839/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14839/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14839/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/14839/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 1s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 13m 25s trunk passed
          +1 compile 0m 15s trunk passed
          +1 checkstyle 0m 12s trunk passed
          +1 mvnsite 0m 16s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 10s trunk passed
          -1 mvninstall 0m 10s hadoop-yarn-server-tests in the patch failed.
          -1 compile 0m 10s hadoop-yarn-server-tests in the patch failed.
          -1 javac 0m 10s hadoop-yarn-server-tests in the patch failed.
          -0 checkstyle 0m 9s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 0 unchanged - 38 fixed = 2 total (was 38)
          -1 mvnsite 0m 11s hadoop-yarn-server-tests in the patch failed.
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 6s the patch passed
          -1 unit 0m 10s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          17m 15s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851568/YARN-6150.005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d60b5f3a78e1 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2007e0c
          Default Java 1.8.0_121
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14861/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14861/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 25s trunk passed +1 compile 0m 15s trunk passed +1 checkstyle 0m 12s trunk passed +1 mvnsite 0m 16s trunk passed +1 mvneclipse 0m 14s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 10s trunk passed -1 mvninstall 0m 10s hadoop-yarn-server-tests in the patch failed. -1 compile 0m 10s hadoop-yarn-server-tests in the patch failed. -1 javac 0m 10s hadoop-yarn-server-tests in the patch failed. -0 checkstyle 0m 9s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 0 unchanged - 38 fixed = 2 total (was 38) -1 mvnsite 0m 11s hadoop-yarn-server-tests in the patch failed. +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 6s the patch passed -1 unit 0m 10s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 17m 15s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851568/YARN-6150.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d60b5f3a78e1 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2007e0c Default Java 1.8.0_121 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14861/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/14861/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 33s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 14m 28s trunk passed
          +1 compile 0m 16s trunk passed
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 17s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 10s trunk passed
          +1 mvninstall 0m 15s the patch passed
          +1 compile 0m 13s the patch passed
          +1 javac 0m 13s the patch passed
          -0 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 27 unchanged - 11 fixed = 29 total (was 38)
          +1 mvnsite 0m 15s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 7s the patch passed
          -1 unit 8m 26s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          27m 15s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851652/YARN-6150.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 686305f29a4b 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / eec52e1
          Default Java 1.8.0_121
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14862/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/14862/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14862/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/14862/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 33s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 28s trunk passed +1 compile 0m 16s trunk passed +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 17s trunk passed +1 mvneclipse 0m 14s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 10s trunk passed +1 mvninstall 0m 15s the patch passed +1 compile 0m 13s the patch passed +1 javac 0m 13s the patch passed -0 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 27 unchanged - 11 fixed = 29 total (was 38) +1 mvnsite 0m 15s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 7s the patch passed -1 unit 8m 26s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 27m 15s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851652/YARN-6150.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 686305f29a4b 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eec52e1 Default Java 1.8.0_121 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14862/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14862/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14862/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/14862/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          rchiang Ray Chiang added a comment -

          Just as an FYI, I'm seeing this fail with regularity on OS X and Linux in trunk with the error:

          java.lang.NullPointerException: null
          at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.waitForContainerToFinishOnNM(TestContainerManagerSecurity.java:398)
          at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.testNMTokens(TestContainerManagerSecurity.java:341)
          at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.testContainerManager(TestContainerManagerSecurity.java:158)

          With this patch, I no longer see the error on either platform.

          Show
          rchiang Ray Chiang added a comment - Just as an FYI, I'm seeing this fail with regularity on OS X and Linux in trunk with the error: java.lang.NullPointerException: null at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.waitForContainerToFinishOnNM(TestContainerManagerSecurity.java:398) at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.testNMTokens(TestContainerManagerSecurity.java:341) at org.apache.hadoop.yarn.server.TestContainerManagerSecurity.testContainerManager(TestContainerManagerSecurity.java:158) With this patch, I no longer see the error on either platform.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 13m 22s trunk passed
          +1 compile 0m 17s trunk passed
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 19s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 11s trunk passed
          +1 mvninstall 0m 15s the patch passed
          +1 compile 0m 14s the patch passed
          +1 javac 0m 14s the patch passed
          -0 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 27 unchanged - 11 fixed = 29 total (was 38)
          +1 mvnsite 0m 15s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 8s the patch passed
          -1 unit 8m 26s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          25m 23s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851652/YARN-6150.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux af73ca3dfb64 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fb68980
          Default Java 1.8.0_131
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16191/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/16191/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16191/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/16191/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 22s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 19s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 11s trunk passed +1 mvninstall 0m 15s the patch passed +1 compile 0m 14s the patch passed +1 javac 0m 14s the patch passed -0 checkstyle 0m 10s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 27 unchanged - 11 fixed = 29 total (was 38) +1 mvnsite 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 8s the patch passed -1 unit 8m 26s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 25m 23s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851652/YARN-6150.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux af73ca3dfb64 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fb68980 Default Java 1.8.0_131 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16191/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/16191/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16191/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/16191/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          rchiang Ray Chiang added a comment -

          Interesting. The failed TestContainerManagerSecurity tests are different. The error in the logs is:

          Failed tests:
          TestContainerManagerSecurity.testContainerManager:167->testNMTokens:268 In calling af73ca3dfb64:49984 exception was 'Invalid host name: local host is: (unknown); destination host is: "af73ca3dfb64":49984; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost' but doesn't contain 'SIMPLE authentication is not enabled. Available:[TOKEN]'
          TestContainerManagerSecurity.testContainerManager:167->testNMTokens:268 In calling af73ca3dfb64:34648 exception was 'Invalid host name: local host is: (unknown); destination host is: "af73ca3dfb64":34648; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost' but doesn't contain 'Client cannot authenticate via:[TOKEN]'

          Show
          rchiang Ray Chiang added a comment - Interesting. The failed TestContainerManagerSecurity tests are different. The error in the logs is: Failed tests: TestContainerManagerSecurity.testContainerManager:167->testNMTokens:268 In calling af73ca3dfb64:49984 exception was 'Invalid host name: local host is: (unknown); destination host is: "af73ca3dfb64":49984; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost ' but doesn't contain 'SIMPLE authentication is not enabled. Available: [TOKEN] ' TestContainerManagerSecurity.testContainerManager:167->testNMTokens:268 In calling af73ca3dfb64:34648 exception was 'Invalid host name: local host is: (unknown); destination host is: "af73ca3dfb64":34648; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost ' but doesn't contain 'Client cannot authenticate via: [TOKEN] '
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 16m 33s trunk passed
          +1 compile 0m 22s trunk passed
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 26s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 13s trunk passed
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 20s the patch passed
          +1 javac 0m 20s the patch passed
          -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 27 unchanged - 11 fixed = 29 total (was 38)
          +1 mvnsite 0m 22s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 11s the patch passed
          -1 unit 8m 52s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          29m 52s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization
            hadoop.yarn.server.TestContainerManagerSecurity



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851652/YARN-6150.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4cadb389def8 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4ebc23b
          Default Java 1.8.0_131
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16194/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/16194/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16194/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/16194/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 16m 33s trunk passed +1 compile 0m 22s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 26s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed -0 checkstyle 0m 12s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 2 new + 27 unchanged - 11 fixed = 29 total (was 38) +1 mvnsite 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 11s the patch passed -1 unit 8m 52s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 29m 52s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization   hadoop.yarn.server.TestContainerManagerSecurity Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851652/YARN-6150.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4cadb389def8 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4ebc23b Default Java 1.8.0_131 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16194/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/16194/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16194/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/16194/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Umm... The error seems that the environment cannot resolve "af73ca3dfb64". I used start-build-env.sh and reproduced the error.

              final InetSocketAddress addr =
                  NetUtils.createSocketAddr(nodeId.getHost(), nodeId.getPort());
          

          In this code, the addr.getAddress() is null in the Docker container.

          Show
          ajisakaa Akira Ajisaka added a comment - Umm... The error seems that the environment cannot resolve "af73ca3dfb64". I used start-build-env.sh and reproduced the error. final InetSocketAddress addr = NetUtils.createSocketAddr(nodeId.getHost(), nodeId.getPort()); In this code, the addr.getAddress() is null in the Docker container.
          Hide
          ajisakaa Akira Ajisaka added a comment - - edited

          If the hostname does not contains dot or is not localhost, NetUtils.createSocketAddr calls SecurityUtil.getByNameWithSearch(String) and it adds trailing dot to the hostname (please see Javadoc of SecurityUtil.getByExactName for the detail) and then resolve the hostname. In the environment, "af73ca3dfb64." (with trailing dot) cannot be resolved, so the error occurs.

          Replacing the above code with

          TestContainerManagerSecurity#getContainerManagementProtocolProxy
              final InetSocketAddress addr =
                  new InetSocketAddress(nodeId.getHost(), nodeId.getPort());
          

          fixes the error.

          Show
          ajisakaa Akira Ajisaka added a comment - - edited If the hostname does not contains dot or is not localhost, NetUtils.createSocketAddr calls SecurityUtil.getByNameWithSearch(String) and it adds trailing dot to the hostname (please see Javadoc of SecurityUtil.getByExactName for the detail) and then resolve the hostname. In the environment, "af73ca3dfb64." (with trailing dot) cannot be resolved, so the error occurs. Replacing the above code with TestContainerManagerSecurity#getContainerManagementProtocolProxy final InetSocketAddress addr = new InetSocketAddress(nodeId.getHost(), nodeId.getPort()); fixes the error.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          or we can fix the error by setting the domain name using -h option when executing docker run.
          https://github.com/moby/moby/issues/9416

          Show
          ajisakaa Akira Ajisaka added a comment - or we can fix the error by setting the domain name using -h option when executing docker run. https://github.com/moby/moby/issues/9416
          Hide
          rchiang Ray Chiang added a comment -

          Thanks for digging into the issue Akira Ajisaka. I have a slight preference for the first solution, since that would make the test a bit more robust.

          Show
          rchiang Ray Chiang added a comment - Thanks for digging into the issue Akira Ajisaka . I have a slight preference for the first solution, since that would make the test a bit more robust.
          Hide
          rchiang Ray Chiang added a comment -

          Daniel Sturman, please implement Akira's suggested code change. That looks to be the last issue.

          Show
          rchiang Ray Chiang added a comment - Daniel Sturman , please implement Akira's suggested code change. That looks to be the last issue.
          Hide
          rchiang Ray Chiang added a comment -

          Adding Akira's suggested change.

          Show
          rchiang Ray Chiang added a comment - Adding Akira's suggested change.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 12m 58s trunk passed
          +1 compile 0m 14s trunk passed
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 0m 15s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s trunk passed
          +1 javadoc 0m 7s trunk passed
          +1 mvninstall 0m 13s the patch passed
          +1 compile 0m 14s the patch passed
          +1 javac 0m 14s the patch passed
          -0 checkstyle 0m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 3 new + 27 unchanged - 11 fixed = 30 total (was 38)
          +1 mvnsite 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          +1 findbugs 0m 0s the patch passed
          +1 javadoc 0m 6s the patch passed
          -1 unit 3m 14s hadoop-yarn-server-tests in the patch failed.
          +1 asflicense 0m 14s The patch does not generate ASF License warnings.
          19m 16s



          Reason Tests
          Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue YARN-6150
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874927/YARN-6150.007.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 30687a497743 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f99b6d1
          Default Java 1.8.0_131
          checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16266/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/16266/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt
          Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16266/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/16266/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 12m 58s trunk passed +1 compile 0m 14s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 15s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s trunk passed +1 javadoc 0m 7s trunk passed +1 mvninstall 0m 13s the patch passed +1 compile 0m 14s the patch passed +1 javac 0m 14s the patch passed -0 checkstyle 0m 8s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests: The patch generated 3 new + 27 unchanged - 11 fixed = 30 total (was 38) +1 mvnsite 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests +1 findbugs 0m 0s the patch passed +1 javadoc 0m 6s the patch passed -1 unit 3m 14s hadoop-yarn-server-tests in the patch failed. +1 asflicense 0m 14s The patch does not generate ASF License warnings. 19m 16s Reason Tests Failed junit tests hadoop.yarn.server.TestMiniYarnClusterNodeUtilization Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue YARN-6150 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874927/YARN-6150.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 30687a497743 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f99b6d1 Default Java 1.8.0_131 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/16266/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/16266/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/16266/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests Console output https://builds.apache.org/job/PreCommit-YARN-Build/16266/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          rchiang Ray Chiang added a comment -

          Unit test failure looks identical to YARN-5728.

          Show
          rchiang Ray Chiang added a comment - Unit test failure looks identical to YARN-5728 .
          Hide
          rchiang Ray Chiang added a comment -

          I thought I had done this before, but the latest version of this patch doesn't seem to fix the unit test in branch-2.

          Show
          rchiang Ray Chiang added a comment - I thought I had done this before, but the latest version of this patch doesn't seem to fix the unit test in branch-2.
          Hide
          Sonia Sonia Garudi added a comment -

          Ray Chiang I have tested the latest patch and the tests pass with the changes made.

          Show
          Sonia Sonia Garudi added a comment - Ray Chiang I have tested the latest patch and the tests pass with the changes made.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I'm +1 for the latest patch for trunk. Now I'm investigating why the test fails in branch-2.

          Show
          ajisakaa Akira Ajisaka added a comment - I'm +1 for the latest patch for trunk. Now I'm investigating why the test fails in branch-2.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          In branch-2, the following error messages are shown many times:

          2017-07-24 15:13:22,921 ERROR [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:run(883)) - Caught exception in status-updater
          java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.yarn.server.api.protocolrecords.NodeHeartbeatResponse, but class was expected
                  at org.apache.hadoop.yarn.server.utils.YarnServerBuilderUtils.newNodeHeartbeatResponse(YarnServerBuilderUtils.java:56)
                  at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.nodeHeartbeat(ResourceTrackerService.java:539)
                  at org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager$1$1.nodeHeartbeat(MiniYARNCluster.java:684)
                  at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:793)
                  at java.lang.Thread.run(Thread.java:745)
          

          YARN-3732 changed NodeHeartbeatResponse from interface to abstract class, so the error does not happen. I cherry-picked YARN-3732 to branch-2 locally and the test passed.

          Show
          ajisakaa Akira Ajisaka added a comment - In branch-2, the following error messages are shown many times: 2017-07-24 15:13:22,921 ERROR [Node Status Updater] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:run(883)) - Caught exception in status-updater java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.yarn.server.api.protocolrecords.NodeHeartbeatResponse, but class was expected at org.apache.hadoop.yarn.server.utils.YarnServerBuilderUtils.newNodeHeartbeatResponse(YarnServerBuilderUtils.java:56) at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.nodeHeartbeat(ResourceTrackerService.java:539) at org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager$1$1.nodeHeartbeat(MiniYARNCluster.java:684) at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl$1.run(NodeStatusUpdaterImpl.java:793) at java.lang.Thread.run(Thread.java:745) YARN-3732 changed NodeHeartbeatResponse from interface to abstract class, so the error does not happen. I cherry-picked YARN-3732 to branch-2 locally and the test passed.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Finally I didn't need to cherry-pick YARN-3732. I ran mvn clean and the test passed.

          Show
          ajisakaa Akira Ajisaka added a comment - Finally I didn't need to cherry-pick YARN-3732 . I ran mvn clean and the test passed.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I'll commit this to trunk and branch-2 tomorrow if there are no objections.

          Show
          ajisakaa Akira Ajisaka added a comment - I'll commit this to trunk and branch-2 tomorrow if there are no objections.
          Hide
          rchiang Ray Chiang added a comment -

          +1

          Thanks Akira Ajisaka for digging into this.

          Show
          rchiang Ray Chiang added a comment - +1 Thanks Akira Ajisaka for digging into this.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk and branch-2. Thanks Daniel Sturman and Ray Chiang for the contribution and thanks Daniel Templeton for the review.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk and branch-2. Thanks Daniel Sturman and Ray Chiang for the contribution and thanks Daniel Templeton for the review.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12052 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12052/)
          YARN-6150. TestContainerManagerSecurity tests for Yarn Server are (aajisaka: rev 218b1b33ffe83cf2e330a2aa90685d0c14547a3d)

          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12052 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12052/ ) YARN-6150 . TestContainerManagerSecurity tests for Yarn Server are (aajisaka: rev 218b1b33ffe83cf2e330a2aa90685d0c14547a3d) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java

            People

            • Assignee:
              sturman Daniel Sturman
              Reporter:
              sturman Daniel Sturman
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development