Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5694

ZKRMStateStore can prevent the transition to standby in branch-2.7 if the ZK node is unreachable

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      ZKRMStateStore.doStoreMultiWithRetries() holds the lock while trying to talk to ZK. If the connection fails, it will retry while still holding the lock. The retries are intended to be strictly time limited, but in the case that the ZK node is unreachable, the time limit fails, resulting in the thread holding the lock for over an hour. Transitioning the RM to standby requires that same lock, so in exactly the case that the RM should be transitioning to standby, the VerifyActiveStatusThread blocks it from happening.

      1. YARN-5694.001.patch
        1 kB
        Daniel Templeton
      2. YARN-5694.002.patch
        2 kB
        Daniel Templeton
      3. YARN-5694.003.patch
        2 kB
        Daniel Templeton
      4. YARN-5694.004.patch
        12 kB
        Daniel Templeton
      5. YARN-5694.004.patch
        12 kB
        Daniel Templeton
      6. YARN-5694.005.patch
        12 kB
        Daniel Templeton
      7. YARN-5694.006.patch
        12 kB
        Daniel Templeton
      8. YARN-5694.007.patch
        12 kB
        Daniel Templeton
      9. YARN-5694.008.patch
        1 kB
        Daniel Templeton
      10. YARN-5694.branch-2.6.001.patch
        7 kB
        Daniel Templeton
      11. YARN-5694.branch-2.6.002.patch
        7 kB
        Daniel Templeton
      12. YARN-5694.branch-2.7.001.patch
        1 kB
        Daniel Templeton
      13. YARN-5694.branch-2.7.002.patch
        14 kB
        Daniel Templeton
      14. YARN-5694.branch-2.7.004.patch
        8 kB
        Daniel Templeton
      15. YARN-5694.branch-2.7.005.patch
        7 kB
        Daniel Templeton

        Activity

        Hide
        templedf Daniel Templeton added a comment -

        Here are patches for trunk and 2.7.

        Show
        templedf Daniel Templeton added a comment - Here are patches for trunk and 2.7.
        Hide
        kasha Karthik Kambatla added a comment -

        We should probably check for isAutoFailoverEnabledAndEmbedded?

        I filed YARN-5709 to clean up surrounding code. Please feel free to pick that up.

        Show
        kasha Karthik Kambatla added a comment - We should probably check for isAutoFailoverEnabledAndEmbedded? I filed YARN-5709 to clean up surrounding code. Please feel free to pick that up.
        Hide
        templedf Daniel Templeton added a comment -

        Based on offline conversation, it sounds like the enabled/embedded check should be removed as it rules out the use case of having a different ZK instance for the leader election as for the state store. Did I understand that correctly, Karthik Kambatla?

        Show
        templedf Daniel Templeton added a comment - Based on offline conversation, it sounds like the enabled/embedded check should be removed as it rules out the use case of having a different ZK instance for the leader election as for the state store. Did I understand that correctly, Karthik Kambatla ?
        Hide
        kasha Karthik Kambatla added a comment -

        Yes. We shouldn't check for the type of leader election we use.

        Should we run the VerifyActiveStatus thread even if HA is not enabled? What if a user configures two RMs to use the same store but forgets to configure HA?

        Show
        kasha Karthik Kambatla added a comment - Yes. We shouldn't check for the type of leader election we use. Should we run the VerifyActiveStatus thread even if HA is not enabled? What if a user configures two RMs to use the same store but forgets to configure HA?
        Hide
        templedf Daniel Templeton added a comment -

        These patches make the verify status thread non-optional and also resolve a locking issue when stopping the service. I'm not sure what to do about tests yet...

        Show
        templedf Daniel Templeton added a comment - These patches make the verify status thread non-optional and also resolve a locking issue when stopping the service. I'm not sure what to do about tests yet...
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 9s YARN-5694 does not apply to branch-2.7. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833837/YARN-5694.branch-2.7.002.patch
        JIRA Issue YARN-5694
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13415/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 9s YARN-5694 does not apply to branch-2.7. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833837/YARN-5694.branch-2.7.002.patch JIRA Issue YARN-5694 Console output https://builds.apache.org/job/PreCommit-YARN-Build/13415/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Looks like I got my patches crossed. Let's try this again.

        Show
        templedf Daniel Templeton added a comment - Looks like I got my patches crossed. Let's try this again.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 27s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 11s branch-2.7 passed
        +1 compile 0m 24s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_111
        +1 checkstyle 0m 21s branch-2.7 passed
        +1 mvnsite 0m 34s branch-2.7 passed
        +1 mvneclipse 0m 17s branch-2.7 passed
        -1 findbugs 1m 2s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings.
        +1 javadoc 0m 18s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_111
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 22s the patch passed with JDK v1.8.0_101
        +1 javac 0m 22s the patch passed
        +1 compile 0m 25s the patch passed with JDK v1.7.0_111
        +1 javac 0m 25s the patch passed
        -1 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 227 unchanged - 4 fixed = 228 total (was 231)
        +1 mvnsite 0m 31s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 1290 line(s) that end in whitespace. Use git apply --whitespace=fix.
        -1 whitespace 0m 34s The patch 70 line(s) with tabs.
        +1 findbugs 1m 11s the patch passed
        +1 javadoc 0m 16s the patch passed with JDK v1.8.0_101
        +1 javadoc 0m 21s the patch passed with JDK v1.7.0_111
        -1 unit 49m 26s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101.
        -1 unit 51m 32s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_111.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        131m 37s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
        JDK v1.7.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833843/YARN-5694.branch-2.7.003.patch
        JIRA Issue YARN-5694
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 048e5886fa9e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 8eb0b6f
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_111.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_111.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13416/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13416/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 11s branch-2.7 passed +1 compile 0m 24s branch-2.7 passed with JDK v1.8.0_101 +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_111 +1 checkstyle 0m 21s branch-2.7 passed +1 mvnsite 0m 34s branch-2.7 passed +1 mvneclipse 0m 17s branch-2.7 passed -1 findbugs 1m 2s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 18s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_111 +1 mvninstall 0m 27s the patch passed +1 compile 0m 22s the patch passed with JDK v1.8.0_101 +1 javac 0m 22s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_111 +1 javac 0m 25s the patch passed -1 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 227 unchanged - 4 fixed = 228 total (was 231) +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 1290 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 whitespace 0m 34s The patch 70 line(s) with tabs. +1 findbugs 1m 11s the patch passed +1 javadoc 0m 16s the patch passed with JDK v1.8.0_101 +1 javadoc 0m 21s the patch passed with JDK v1.7.0_111 -1 unit 49m 26s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_101. -1 unit 51m 32s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_111. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 131m 37s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833843/YARN-5694.branch-2.7.003.patch JIRA Issue YARN-5694 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 048e5886fa9e 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 8eb0b6f Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_111.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_101.txt https://builds.apache.org/job/PreCommit-YARN-Build/13416/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13416/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13416/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Looks like the Jenkins complaints are all invalid. The unit tests pass for me locally, and the rest is unrelated.

        Show
        templedf Daniel Templeton added a comment - Looks like the Jenkins complaints are all invalid. The unit tests pass for me locally, and the rest is unrelated.
        Hide
        templedf Daniel Templeton added a comment -

        New patch that includes tests. It also changes the RM's behavior a bit, per the discussion with Karthik Kambatla above. Now, if the RM is not in HA mode and is forced into standby mode, it will exit to prevent potential state store corruption. The only scenario where this can happen is when the ZKRMStateStore discovers that it has been fenced (not when it has lost contact with the ZK instance), which is exactly when exiting is the right thing to do.

        Show
        templedf Daniel Templeton added a comment - New patch that includes tests. It also changes the RM's behavior a bit, per the discussion with Karthik Kambatla above. Now, if the RM is not in HA mode and is forced into standby mode, it will exit to prevent potential state store corruption. The only scenario where this can happen is when the ZKRMStateStore discovers that it has been fenced (not when it has lost contact with the ZK instance), which is exactly when exiting is the right thing to do.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 45s trunk passed
        +1 compile 0m 32s trunk passed
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 37s trunk passed
        +1 mvneclipse 0m 17s trunk passed
        +1 findbugs 0m 56s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 30s the patch passed
        +1 compile 0m 29s the patch passed
        +1 javac 0m 29s the patch passed
        -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 115 unchanged - 3 fixed = 117 total (was 118)
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 1s the patch passed
        +1 javadoc 0m 17s the patch passed
        -1 unit 54m 37s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        69m 5s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
          hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834049/YARN-5694.004.patch
        JIRA Issue YARN-5694
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f69a1ffc76c0 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c62ae71
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13432/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13432/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13432/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13432/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13432/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 45s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 37s trunk passed +1 mvneclipse 0m 17s trunk passed +1 findbugs 0m 56s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 30s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed -1 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 2 new + 115 unchanged - 3 fixed = 117 total (was 118) +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 1s the patch passed +1 javadoc 0m 17s the patch passed -1 unit 54m 37s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 69m 5s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestContainerResourceUsage   hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834049/YARN-5694.004.patch JIRA Issue YARN-5694 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f69a1ffc76c0 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c62ae71 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13432/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13432/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13432/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13432/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13432/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        I addressed the line length issue in the checkstyle report. The unused import isn't actually unused. The unit test failures are unrelated.

        Show
        templedf Daniel Templeton added a comment - I addressed the line length issue in the checkstyle report. The unused import isn't actually unused. The unit test failures are unrelated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 47s trunk passed
        +1 compile 0m 32s trunk passed
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 38s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 57s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 31s the patch passed
        +1 compile 0m 29s the patch passed
        +1 javac 0m 29s the patch passed
        -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 115 unchanged - 3 fixed = 116 total (was 118)
        +1 mvnsite 0m 34s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 4s the patch passed
        +1 javadoc 0m 17s the patch passed
        +1 unit 35m 6s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        49m 35s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834163/YARN-5694.005.patch
        JIRA Issue YARN-5694
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1ce877c55ba2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c5573e6
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13438/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13438/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13438/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 47s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 38s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 57s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 31s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed -1 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 115 unchanged - 3 fixed = 116 total (was 118) +1 mvnsite 0m 34s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 4s the patch passed +1 javadoc 0m 17s the patch passed +1 unit 35m 6s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 49m 35s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12834163/YARN-5694.005.patch JIRA Issue YARN-5694 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1ce877c55ba2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c5573e6 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13438/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13438/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13438/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Again, the unused import isn't actually unused.

        Show
        templedf Daniel Templeton added a comment - Again, the unused import isn't actually unused.
        Hide
        kasha Karthik Kambatla added a comment -

        I have minor comments (nits) on the trunk patch:

        1. ResourceManager: Since the method is now not just transitioning to standby, it should likely be named differently. How about handleFatalFailure()?
        2. ZKRMStateStore: In closeInternal, I would still prefer the null check on verifyActiveStatusThread.

        On the branch-2.7 patch, does it also include the YARN-5677 patch?

        Show
        kasha Karthik Kambatla added a comment - I have minor comments (nits) on the trunk patch: ResourceManager: Since the method is now not just transitioning to standby, it should likely be named differently. How about handleFatalFailure() ? ZKRMStateStore: In closeInternal, I would still prefer the null check on verifyActiveStatusThread. On the branch-2.7 patch, does it also include the YARN-5677 patch?
        Hide
        templedf Daniel Templeton added a comment -

        Here's a patch to add the null check back. I don't think we need to rename the handleTransitionToStandby() method, because the exit happens only if we can't transition to standby, i.e. we're not in HA mode. To my way of thinking, that doesn't change the purpose of the method.

        Show
        templedf Daniel Templeton added a comment - Here's a patch to add the null check back. I don't think we need to rename the handleTransitionToStandby() method, because the exit happens only if we can't transition to standby, i.e. we're not in HA mode. To my way of thinking, that doesn't change the purpose of the method.
        Hide
        templedf Daniel Templeton added a comment -

        The branch-2.7 patch is out of date. I stopped updating it until we settle on the trunk patch.

        Show
        templedf Daniel Templeton added a comment - The branch-2.7 patch is out of date. I stopped updating it until we settle on the trunk patch.
        Hide
        kasha Karthik Kambatla added a comment -

        I am not very particular about the method name, but do think the current name does not fully capture the intention.

        +1, pending Jenkins. Just manually kick started it.

        Show
        kasha Karthik Kambatla added a comment - I am not very particular about the method name, but do think the current name does not fully capture the intention. +1, pending Jenkins. Just manually kick started it.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        -1 mvninstall 2m 48s root in trunk failed.
        +1 compile 0m 38s trunk passed
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 44s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 0m 58s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 34s the patch passed
        +1 compile 0m 30s the patch passed
        +1 javac 0m 30s the patch passed
        -0 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 115 unchanged - 3 fixed = 116 total (was 118)
        +1 mvnsite 0m 35s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 2s the patch passed
        +1 javadoc 0m 19s the patch passed
        -1 unit 38m 39s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        50m 25s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue YARN-5694
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835196/YARN-5694.006.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f83c5f3087ac 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2528bea
        Default Java 1.8.0_101
        mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13706/artifact/patchprocess/branch-mvninstall-root.txt
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13706/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13706/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13706/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13706/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. -1 mvninstall 2m 48s root in trunk failed. +1 compile 0m 38s trunk passed +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 44s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 58s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 34s the patch passed +1 compile 0m 30s the patch passed +1 javac 0m 30s the patch passed -0 checkstyle 0m 19s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 115 unchanged - 3 fixed = 116 total (was 118) +1 mvnsite 0m 35s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 2s the patch passed +1 javadoc 0m 19s the patch passed -1 unit 38m 39s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 50m 25s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-5694 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835196/YARN-5694.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f83c5f3087ac 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2528bea Default Java 1.8.0_101 mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/13706/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13706/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13706/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13706/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13706/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Karthik Kambatla, to meet you half way on the method name, I added javadoc that explains everything clearly.

        Show
        templedf Daniel Templeton added a comment - Karthik Kambatla , to meet you half way on the method name, I added javadoc that explains everything clearly.
        Hide
        kasha Karthik Kambatla added a comment -

        +1, pending Jenkins.

        Show
        kasha Karthik Kambatla added a comment - +1, pending Jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 9m 2s trunk passed
        +1 compile 0m 43s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 23s trunk passed
        +1 findbugs 1m 18s trunk passed
        +1 javadoc 0m 25s trunk passed
        +1 mvninstall 0m 37s the patch passed
        +1 compile 0m 37s the patch passed
        +1 javac 0m 37s the patch passed
        -0 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 115 unchanged - 3 fixed = 116 total (was 118)
        +1 mvnsite 0m 50s the patch passed
        +1 mvneclipse 0m 17s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 17s the patch passed
        +1 javadoc 0m 19s the patch passed
        -1 unit 36m 11s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        55m 33s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
          hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue YARN-5694
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836393/YARN-5694.007.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1af4e42b6702 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 34173a4
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13738/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13738/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13738/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13738/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 2s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 1m 18s trunk passed +1 javadoc 0m 25s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 0m 37s the patch passed +1 javac 0m 37s the patch passed -0 checkstyle 0m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 115 unchanged - 3 fixed = 116 total (was 118) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 17s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 17s the patch passed +1 javadoc 0m 19s the patch passed -1 unit 36m 11s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 55m 33s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue YARN-5694 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12836393/YARN-5694.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1af4e42b6702 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 34173a4 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/13738/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/13738/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13738/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13738/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        One of the unit test failures is YARN-5043. The other looks a little like YARN-5057, except that YARN-5057 was already fixed. I was able to reproduce the test failure without my patch applied, so I filed YARN-5816 for it.

        Show
        templedf Daniel Templeton added a comment - One of the unit test failures is YARN-5043 . The other looks a little like YARN-5057 , except that YARN-5057 was already fixed. I was able to reproduce the test failure without my patch applied, so I filed YARN-5816 for it.
        Hide
        jianhe Jian He added a comment - - edited

        sorry, I don't quite get it. why is the active thread needed to run in non-HA mode ?
        Even in HA mode, it may not be needed, because the curator leader election library can automatically detect whether the RM is still active and send notification. IIUC, no need a separate thread to detect that

        Show
        jianhe Jian He added a comment - - edited sorry, I don't quite get it. why is the active thread needed to run in non-HA mode ? Even in HA mode, it may not be needed, because the curator leader election library can automatically detect whether the RM is still active and send notification. IIUC, no need a separate thread to detect that
        Hide
        templedf Daniel Templeton added a comment -

        The concern is that the leader election and state store can be configured to use different ZK instances in HA mode. In that case, the state store still has to protect itself. In non-HA, it may still be possible for a second RM to start using the same cluster ID and same ZK instance, which would corrupt the state store. By having the state store be always vigilant, we protect ourselves from state store corruption in all cases.

        Show
        templedf Daniel Templeton added a comment - The concern is that the leader election and state store can be configured to use different ZK instances in HA mode. In that case, the state store still has to protect itself. In non-HA, it may still be possible for a second RM to start using the same cluster ID and same ZK instance, which would corrupt the state store. By having the state store be always vigilant, we protect ourselves from state store corruption in all cases.
        Hide
        jianhe Jian He added a comment -

        One other thing is the normal state-store operation already piggybacks a create and delete fencing node operations. In that case, it's already covered that if the state-store is fenced, the store operation will not succeed.

        Show
        jianhe Jian He added a comment - One other thing is the normal state-store operation already piggybacks a create and delete fencing node operations. In that case, it's already covered that if the state-store is fenced, the store operation will not succeed.
        Hide
        templedf Daniel Templeton added a comment -

        Yes, but if the RM isn't in HA mode, the fencing is quietly ignored, which is also something I should address in the next version of this patch.

        The reason to have the thread always run is so that we react earlier. If we agree that it's bad to have two RMs accidentally sharing the same state store, why would you not want to catch the issue as early as possible?

        Show
        templedf Daniel Templeton added a comment - Yes, but if the RM isn't in HA mode, the fencing is quietly ignored, which is also something I should address in the next version of this patch. The reason to have the thread always run is so that we react earlier. If we agree that it's bad to have two RMs accidentally sharing the same state store, why would you not want to catch the issue as early as possible?
        Hide
        jianhe Jian He added a comment - - edited

        If we agree that it's bad to have two RMs accidentally sharing the same state store,

        If it's in non-HA mode, currently there's no protection in the ZKStore preventing two RMs from sharing the same store. All the ACLs setting related code is only used in HA mode. Essentially, with current patch, I doubt it will get NoAuthException in the verifyThread, without making user change the ACLs manually. So the handling code in this patch will not be triggered with default setting. Maybe I'm wrong, you may try on a real cluster.. also, I thinking setting ACLs for RM is not a required step for deploying non-HA cluster, forcing this to be set is behavior change..

        why would you not want to catch the issue as early as possible?

        My point is that first,will this code work as mentioned above. second, if there's no difference in terms of functionality, why do I need to start a thread pinging the zk continuously every few seconds. Of course, I might miss something, you may clarify more...

        Also, is the use-case mainly about two clusters sharing the same zk-store with the same path ? IMHO, this is not a primary use-case to solve, if user mis-configured, it's user's fault. There are many other places that can go wrong. e.g. if two clusters configure the same path for anything on HDFS.

        If the use-case is about two RMs sharing the same zk-path in the same cluster with non-HA mode. I think in non-HA mode, the invalid RM will not take workload in the first place, clients, NMs will not switch to that RM if HA is not configured properly.

        Show
        jianhe Jian He added a comment - - edited If we agree that it's bad to have two RMs accidentally sharing the same state store, If it's in non-HA mode, currently there's no protection in the ZKStore preventing two RMs from sharing the same store. All the ACLs setting related code is only used in HA mode. Essentially, with current patch, I doubt it will get NoAuthException in the verifyThread, without making user change the ACLs manually. So the handling code in this patch will not be triggered with default setting. Maybe I'm wrong, you may try on a real cluster.. also, I thinking setting ACLs for RM is not a required step for deploying non-HA cluster, forcing this to be set is behavior change.. why would you not want to catch the issue as early as possible? My point is that first,will this code work as mentioned above. second, if there's no difference in terms of functionality, why do I need to start a thread pinging the zk continuously every few seconds. Of course, I might miss something, you may clarify more... Also, is the use-case mainly about two clusters sharing the same zk-store with the same path ? IMHO, this is not a primary use-case to solve, if user mis-configured, it's user's fault. There are many other places that can go wrong. e.g. if two clusters configure the same path for anything on HDFS. If the use-case is about two RMs sharing the same zk-path in the same cluster with non-HA mode. I think in non-HA mode, the invalid RM will not take workload in the first place, clients, NMs will not switch to that RM if HA is not configured properly.
        Hide
        templedf Daniel Templeton added a comment -

        Fair points. I see two ways forward. Either I make all the HA-only behavior always enabled, which will protect the RM in the event of a second cluster being misconfigured to use the first cluster's state store, or I drop this JIRA back to its original purpose, which was fixing the conditional in ZKRMStateStore.startInternal() and dropping the synchronized from closeInternal(). I'll post a patch for the latter, and we can discuss whether it's worth filing another issue for the former.

        Show
        templedf Daniel Templeton added a comment - Fair points. I see two ways forward. Either I make all the HA-only behavior always enabled, which will protect the RM in the event of a second cluster being misconfigured to use the first cluster's state store, or I drop this JIRA back to its original purpose, which was fixing the conditional in ZKRMStateStore.startInternal() and dropping the synchronized from closeInternal() . I'll post a patch for the latter, and we can discuss whether it's worth filing another issue for the former.
        Hide
        templedf Daniel Templeton added a comment - - edited

        This patch turns the active status thread back on whenever HA is on. (Currently, it's only started with manual failover, which doesn't make any sense.) This patch also removes the synchronization from closeInternal() because it causes the transition to standby to hang if the active status thread gets hung up, such as when the ZK node goes dark.

        Show
        templedf Daniel Templeton added a comment - - edited This patch turns the active status thread back on whenever HA is on. (Currently, it's only started with manual failover, which doesn't make any sense.) This patch also removes the synchronization from closeInternal() because it causes the transition to standby to hang if the active status thread gets hung up, such as when the ZK node goes dark.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 23s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 16s trunk passed
        +1 compile 0m 45s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 47s trunk passed
        +1 mvneclipse 0m 21s trunk passed
        +1 findbugs 1m 15s trunk passed
        +1 javadoc 0m 26s trunk passed
        +1 mvninstall 0m 42s the patch passed
        +1 compile 0m 37s the patch passed
        +1 javac 0m 37s the patch passed
        +1 checkstyle 0m 20s the patch passed
        +1 mvnsite 0m 43s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 32s the patch passed
        +1 javadoc 0m 25s the patch passed
        -1 unit 48m 17s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        68m 24s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue YARN-5694
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839846/YARN-5694.008.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6b55f5b31229 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 683e0c7
        Default Java 1.8.0_111
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13997/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13997/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13997/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 16s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 47s trunk passed +1 mvneclipse 0m 21s trunk passed +1 findbugs 1m 15s trunk passed +1 javadoc 0m 26s trunk passed +1 mvninstall 0m 42s the patch passed +1 compile 0m 37s the patch passed +1 javac 0m 37s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 0m 43s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 32s the patch passed +1 javadoc 0m 25s the patch passed -1 unit 48m 17s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 68m 24s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue YARN-5694 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839846/YARN-5694.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6b55f5b31229 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 683e0c7 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/13997/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13997/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/13997/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Test failure is YARN-5362. I should probably add some tests, though...

        Show
        templedf Daniel Templeton added a comment - Test failure is YARN-5362 . I should probably add some tests, though...
        Hide
        jianhe Jian He added a comment - - edited

        (Currently, it's only started with manual failover, which doesn't make any sense.)

        IIRC, it's started only with manual failover because, in case of curator based leader elector, the curator library will trigger notification already if RM is not active. No need for an additional polling thread... This maybe the case for Hadoop's ActiveStandbyElector too..
        If you think it's better to keep this for Hadoop's ActiveStandbyElector , maybe we can do something like: if (HA.isEnabled() && !curatorEnabled)

        Show
        jianhe Jian He added a comment - - edited (Currently, it's only started with manual failover, which doesn't make any sense.) IIRC, it's started only with manual failover because, in case of curator based leader elector, the curator library will trigger notification already if RM is not active. No need for an additional polling thread... This maybe the case for Hadoop's ActiveStandbyElector too.. If you think it's better to keep this for Hadoop's ActiveStandbyElector , maybe we can do something like: if (HA.isEnabled() && !curatorEnabled)
        Hide
        templedf Daniel Templeton added a comment -

        Jian He, looks like I misunderstood the intended behavior of RM HA. While I go reevaluate that part of this JIRA, I've reduced the scope down to only removing the locking conflict. I also added tests. The locking issue only occurs in branch-2.7 and earlier.

        Show
        templedf Daniel Templeton added a comment - Jian He , looks like I misunderstood the intended behavior of RM HA. While I go reevaluate that part of this JIRA, I've reduced the scope down to only removing the locking conflict. I also added tests. The locking issue only occurs in branch-2.7 and earlier.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 12m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 5s branch-2.7 passed
        +1 compile 0m 24s branch-2.7 passed with JDK v1.8.0_111
        +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_121
        +1 checkstyle 0m 19s branch-2.7 passed
        +1 mvnsite 0m 33s branch-2.7 passed
        +1 mvneclipse 0m 16s branch-2.7 passed
        +1 findbugs 0m 59s branch-2.7 passed
        +1 javadoc 0m 18s branch-2.7 passed with JDK v1.8.0_111
        +1 javadoc 0m 22s branch-2.7 passed with JDK v1.7.0_121
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 21s the patch passed with JDK v1.8.0_111
        +1 javac 0m 21s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_121
        +1 javac 0m 26s the patch passed
        -0 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 229 unchanged - 2 fixed = 232 total (was 231)
        +1 mvnsite 0m 36s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 1s The patch has 3080 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 13s The patch 74 line(s) with tabs.
        +1 findbugs 1m 18s the patch passed
        +1 javadoc 0m 17s the patch passed with JDK v1.8.0_111
        +1 javadoc 0m 22s the patch passed with JDK v1.7.0_121
        -1 unit 52m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        131m 30s



        Reason Tests
        JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
        JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Issue YARN-5694
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840077/YARN-5694.branch-2.7.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 46371b2c0fee 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 01ad9de
        Default Java 1.7.0_121
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt
        JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14027/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14027/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 5s branch-2.7 passed +1 compile 0m 24s branch-2.7 passed with JDK v1.8.0_111 +1 compile 0m 28s branch-2.7 passed with JDK v1.7.0_121 +1 checkstyle 0m 19s branch-2.7 passed +1 mvnsite 0m 33s branch-2.7 passed +1 mvneclipse 0m 16s branch-2.7 passed +1 findbugs 0m 59s branch-2.7 passed +1 javadoc 0m 18s branch-2.7 passed with JDK v1.8.0_111 +1 javadoc 0m 22s branch-2.7 passed with JDK v1.7.0_121 +1 mvninstall 0m 27s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_111 +1 javac 0m 21s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_121 +1 javac 0m 26s the patch passed -0 checkstyle 0m 17s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 229 unchanged - 2 fixed = 232 total (was 231) +1 mvnsite 0m 36s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 1s The patch has 3080 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 13s The patch 74 line(s) with tabs. +1 findbugs 1m 18s the patch passed +1 javadoc 0m 17s the patch passed with JDK v1.8.0_111 +1 javadoc 0m 22s the patch passed with JDK v1.7.0_121 -1 unit 52m 30s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 131m 30s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue YARN-5694 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840077/YARN-5694.branch-2.7.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 46371b2c0fee 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 01ad9de Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14027/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14027/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/14027/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Switched to MockRM in the tests

        Show
        templedf Daniel Templeton added a comment - Switched to MockRM in the tests
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 46s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 7s branch-2.7 passed
        +1 compile 0m 24s branch-2.7 passed with JDK v1.8.0_111
        +1 compile 0m 29s branch-2.7 passed with JDK v1.7.0_121
        +1 checkstyle 0m 20s branch-2.7 passed
        +1 mvnsite 0m 35s branch-2.7 passed
        +1 mvneclipse 0m 16s branch-2.7 passed
        +1 findbugs 1m 3s branch-2.7 passed
        +1 javadoc 0m 19s branch-2.7 passed with JDK v1.8.0_111
        +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_121
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.8.0_111
        +1 javac 0m 23s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_121
        +1 javac 0m 26s the patch passed
        -0 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 229 unchanged - 2 fixed = 232 total (was 231)
        +1 mvnsite 0m 33s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 3402 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 30s The patch 74 line(s) with tabs.
        +1 findbugs 1m 12s the patch passed
        +1 javadoc 0m 16s the patch passed with JDK v1.8.0_111
        +1 javadoc 0m 21s the patch passed with JDK v1.7.0_121
        -1 unit 50m 13s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        128m 22s



        Reason Tests
        JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens
        JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.TestClientRMTokens



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Issue YARN-5694
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840127/YARN-5694.branch-2.7.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c6a696471e7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 48a2dac
        Default Java 1.7.0_121
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt
        JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14035/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14035/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 46s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 7s branch-2.7 passed +1 compile 0m 24s branch-2.7 passed with JDK v1.8.0_111 +1 compile 0m 29s branch-2.7 passed with JDK v1.7.0_121 +1 checkstyle 0m 20s branch-2.7 passed +1 mvnsite 0m 35s branch-2.7 passed +1 mvneclipse 0m 16s branch-2.7 passed +1 findbugs 1m 3s branch-2.7 passed +1 javadoc 0m 19s branch-2.7 passed with JDK v1.8.0_111 +1 javadoc 0m 24s branch-2.7 passed with JDK v1.7.0_121 +1 mvninstall 0m 27s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_111 +1 javac 0m 23s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_121 +1 javac 0m 26s the patch passed -0 checkstyle 0m 18s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 229 unchanged - 2 fixed = 232 total (was 231) +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 3402 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 30s The patch 74 line(s) with tabs. +1 findbugs 1m 12s the patch passed +1 javadoc 0m 16s the patch passed with JDK v1.8.0_111 +1 javadoc 0m 21s the patch passed with JDK v1.7.0_121 -1 unit 50m 13s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 128m 22s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.TestClientRMTokens Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue YARN-5694 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840127/YARN-5694.branch-2.7.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c6a696471e7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 48a2dac Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14035/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14035/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/14035/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        The test failures appear to be HADOOP-12687 and YARN-2634. I'm doing some additional testing to make sure the TestClientRMTokens failures really are unrelated. I should know later today.

        Show
        templedf Daniel Templeton added a comment - The test failures appear to be HADOOP-12687 and YARN-2634 . I'm doing some additional testing to make sure the TestClientRMTokens failures really are unrelated. I should know later today.
        Hide
        templedf Daniel Templeton added a comment -

        I ran TestClientRMTokens 500 times with just branch-2.7 and 500 times with branch-2.7 + this patch. In neither case did I encounter any test failures.

        Show
        templedf Daniel Templeton added a comment - I ran TestClientRMTokens 500 times with just branch-2.7 and 500 times with branch-2.7 + this patch. In neither case did I encounter any test failures.
        Hide
        jianhe Jian He added a comment -

        latest patch for branch-2.7 looks good to me, thanks Daniel !
        is this required for 2.6 ?

        Show
        jianhe Jian He added a comment - latest patch for branch-2.7 looks good to me, thanks Daniel ! is this required for 2.6 ?
        Hide
        templedf Daniel Templeton added a comment -

        Thanks, Jian He. Yeah, it probably should go into 2.6 as well.

        Show
        templedf Daniel Templeton added a comment - Thanks, Jian He . Yeah, it probably should go into 2.6 as well.
        Hide
        jianhe Jian He added a comment -

        I see, then we need a separate patch for 2.6 in that case, as current 2.7 patch doesn't apply

        Show
        jianhe Jian He added a comment - I see, then we need a separate patch for 2.6 in that case, as current 2.7 patch doesn't apply
        Hide
        templedf Daniel Templeton added a comment -

        Here's a 2.6 patch.

        Show
        templedf Daniel Templeton added a comment - Here's a 2.6 patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 42s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 52s branch-2.6 passed
        +1 compile 0m 24s branch-2.6 passed with JDK v1.8.0_111
        +1 compile 0m 28s branch-2.6 passed with JDK v1.7.0_121
        +1 checkstyle 0m 17s branch-2.6 passed
        +1 mvnsite 0m 35s branch-2.6 passed
        +1 mvneclipse 0m 16s branch-2.6 passed
        +1 findbugs 0m 57s branch-2.6 passed
        -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in branch-2.6 failed with JDK v1.8.0_111.
        +1 javadoc 0m 22s branch-2.6 passed with JDK v1.7.0_121
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 21s the patch passed with JDK v1.8.0_111
        +1 javac 0m 21s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_121
        +1 javac 0m 26s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 31s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 3765 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 32s The patch 74 line(s) with tabs.
        +1 findbugs 1m 6s the patch passed
        -1 javadoc 0m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_111.
        +1 javadoc 0m 20s the patch passed with JDK v1.7.0_121
        -1 unit 48m 1s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121.
        -1 asflicense 0m 43s The patch generated 127 ASF License warnings.
        125m 18s



        Reason Tests
        JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
          hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
        JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:44eef0e
        JIRA Issue YARN-5694
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840952/YARN-5694.branch-2.6.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 42daee31e0e6 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.6 / c2d936f
        Default Java 1.7.0_121
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
        findbugs v1.3.9
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/whitespace-tabs.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt
        JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14127/testReport/
        asflicense https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14127/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 42s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 52s branch-2.6 passed +1 compile 0m 24s branch-2.6 passed with JDK v1.8.0_111 +1 compile 0m 28s branch-2.6 passed with JDK v1.7.0_121 +1 checkstyle 0m 17s branch-2.6 passed +1 mvnsite 0m 35s branch-2.6 passed +1 mvneclipse 0m 16s branch-2.6 passed +1 findbugs 0m 57s branch-2.6 passed -1 javadoc 0m 19s hadoop-yarn-server-resourcemanager in branch-2.6 failed with JDK v1.8.0_111. +1 javadoc 0m 22s branch-2.6 passed with JDK v1.7.0_121 +1 mvninstall 0m 27s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_111 +1 javac 0m 21s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_121 +1 javac 0m 26s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 3765 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 32s The patch 74 line(s) with tabs. +1 findbugs 1m 6s the patch passed -1 javadoc 0m 14s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_111. +1 javadoc 0m 20s the patch passed with JDK v1.7.0_121 -1 unit 48m 1s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121. -1 asflicense 0m 43s The patch generated 127 ASF License warnings. 125m 18s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore   hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore Subsystem Report/Notes Docker Image:yetus/hadoop:44eef0e JIRA Issue YARN-5694 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12840952/YARN-5694.branch-2.6.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 42daee31e0e6 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.6 / c2d936f Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v1.3.9 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/whitespace-tabs.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14127/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/14127/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/14127/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment - - edited

        The test failure looks legit (which is odd since it worked locally), but the rest of the issues are bogus. I'll take a closer look at the test failure.

        Show
        templedf Daniel Templeton added a comment - - edited The test failure looks legit (which is odd since it worked locally), but the rest of the issues are bogus. I'll take a closer look at the test failure.
        Hide
        templedf Daniel Templeton added a comment -

        Uploading new branch-2.6 patch to fix the test.

        Show
        templedf Daniel Templeton added a comment - Uploading new branch-2.6 patch to fix the test.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 16m 8s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 26s branch-2.6 passed
        +1 compile 0m 25s branch-2.6 passed with JDK v1.8.0_111
        +1 compile 0m 29s branch-2.6 passed with JDK v1.7.0_121
        +1 checkstyle 0m 18s branch-2.6 passed
        +1 mvnsite 0m 35s branch-2.6 passed
        +1 mvneclipse 0m 16s branch-2.6 passed
        +1 findbugs 0m 58s branch-2.6 passed
        -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in branch-2.6 failed with JDK v1.8.0_111.
        +1 javadoc 0m 23s branch-2.6 passed with JDK v1.7.0_121
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.8.0_111
        +1 javac 0m 23s the patch passed
        +1 compile 0m 26s the patch passed with JDK v1.7.0_121
        +1 javac 0m 26s the patch passed
        +1 checkstyle 0m 13s the patch passed
        +1 mvnsite 0m 33s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 4112 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 35s The patch 74 line(s) with tabs.
        +1 findbugs 1m 8s the patch passed
        -1 javadoc 0m 15s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_111.
        +1 javadoc 0m 20s the patch passed with JDK v1.7.0_121
        -1 unit 47m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121.
        -1 asflicense 0m 31s The patch generated 127 ASF License warnings.
        129m 36s



        Reason Tests
        JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
        JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
          hadoop.yarn.server.resourcemanager.TestAMAuthorization
          hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:44eef0e
        JIRA Issue YARN-5694
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841385/YARN-5694.branch-2.6.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b8f7553fe418 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.6 / c2d936f
        Default Java 1.7.0_121
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
        findbugs v1.3.9
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/whitespace-tabs.txt
        javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt
        unit https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt
        JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14153/testReport/
        asflicense https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/14153/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 26s branch-2.6 passed +1 compile 0m 25s branch-2.6 passed with JDK v1.8.0_111 +1 compile 0m 29s branch-2.6 passed with JDK v1.7.0_121 +1 checkstyle 0m 18s branch-2.6 passed +1 mvnsite 0m 35s branch-2.6 passed +1 mvneclipse 0m 16s branch-2.6 passed +1 findbugs 0m 58s branch-2.6 passed -1 javadoc 0m 21s hadoop-yarn-server-resourcemanager in branch-2.6 failed with JDK v1.8.0_111. +1 javadoc 0m 23s branch-2.6 passed with JDK v1.7.0_121 +1 mvninstall 0m 27s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_111 +1 javac 0m 23s the patch passed +1 compile 0m 26s the patch passed with JDK v1.7.0_121 +1 javac 0m 26s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 33s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 4112 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 35s The patch 74 line(s) with tabs. +1 findbugs 1m 8s the patch passed -1 javadoc 0m 15s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_111. +1 javadoc 0m 20s the patch passed with JDK v1.7.0_121 -1 unit 47m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_121. -1 asflicense 0m 31s The patch generated 127 ASF License warnings. 129m 36s Reason Tests JDK v1.8.0_111 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_121 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore Subsystem Report/Notes Docker Image:yetus/hadoop:44eef0e JIRA Issue YARN-5694 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12841385/YARN-5694.branch-2.6.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b8f7553fe418 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.6 / c2d936f Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v1.3.9 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/whitespace-tabs.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_111.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/14153/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/14153/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Console output https://builds.apache.org/job/PreCommit-YARN-Build/14153/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        The warnings, errors, and failures on the branch-2.6 patch are all unrelated. I replicated the TestFSRMStateStore test failure locally without my patch applied. I'll play with the test in trunk and file a JIRA if it's flakey.

        Show
        templedf Daniel Templeton added a comment - The warnings, errors, and failures on the branch-2.6 patch are all unrelated. I replicated the TestFSRMStateStore test failure locally without my patch applied. I'll play with the test in trunk and file a JIRA if it's flakey.
        Hide
        jianhe Jian He added a comment -

        sounds good, i'll commit the 2.7 and 2.6 patches

        Show
        jianhe Jian He added a comment - sounds good, i'll commit the 2.7 and 2.6 patches
        Hide
        jianhe Jian He added a comment -

        committed to branch-2.6 and branch-2.7, thanks Daniel Templeton !

        Show
        jianhe Jian He added a comment - committed to branch-2.6 and branch-2.7, thanks Daniel Templeton !
        Hide
        templedf Daniel Templeton added a comment -

        Thanks for the reviews and commit, Jian He!

        Show
        templedf Daniel Templeton added a comment - Thanks for the reviews and commit, Jian He !
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Jian He can you update changes.txt for branch-2.7

        Show
        brahmareddy Brahma Reddy Battula added a comment - Jian He can you update changes.txt for branch-2.7

          People

          • Assignee:
            templedf Daniel Templeton
            Reporter:
            templedf Daniel Templeton
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development