Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12715

TestValueQueue#testgetAtMostPolicyALL fails intermittently

    Details

    • Type: Test
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The test fails intermittently with the following error.
      Error Message

      expected:<19> but was:<10>
      

      Stacktrace

      java.lang.AssertionError: expected:<19> but was:<10>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.junit.Assert.assertEquals(Assert.java:555)
      	at org.junit.Assert.assertEquals(Assert.java:542)
      	at org.apache.hadoop.crypto.key.TestValueQueue.testgetAtMostPolicyALL(TestValueQueue.java:149)
      
      1. HADOOP-12715.01.patch
        5 kB
        Xiao Chen
      2. HADOOP-12715.02.patch
        5 kB
        Xiao Chen
      3. HADOOP-12715.03.patch
        5 kB
        Xiao Chen

        Activity

        Hide
        xiaochen Xiao Chen added a comment -

        Reason for the failure is because ValueQueue has an async filling thread. There's a race between the async filling after getAtMost(10), and the sync filling of getAtMost(19).

        Patch 1 enhanced the test to first check the async getAtMost(10), then check the sync getAtMost(19). Also added timeout to all tests in this class, and enhanced the asserts to have a message.

        Show
        xiaochen Xiao Chen added a comment - Reason for the failure is because ValueQueue has an async filling thread. There's a race between the async filling after getAtMost(10) , and the sync filling of getAtMost(19) . Patch 1 enhanced the test to first check the async getAtMost(10) , then check the sync getAtMost(19) . Also added timeout to all tests in this class, and enhanced the asserts to have a message.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 9m 30s trunk passed
        -1 compile 6m 2s root in trunk failed with JDK v1.8.0_66.
        -1 compile 0m 47s root in trunk failed with JDK v1.7.0_91.
        +1 checkstyle 0m 19s trunk passed
        +1 mvnsite 1m 8s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 2m 12s trunk passed
        +1 javadoc 1m 12s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 12s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 23s the patch passed
        -1 compile 4m 7s root in the patch failed with JDK v1.8.0_66.
        -1 javac 4m 7s root in the patch failed with JDK v1.8.0_66.
        -1 compile 0m 57s root in the patch failed with JDK v1.7.0_91.
        -1 javac 0m 57s root in the patch failed with JDK v1.7.0_91.
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 1m 19s the patch passed
        +1 mvneclipse 0m 18s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 3m 9s the patch passed
        +1 javadoc 1m 47s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 33s the patch passed with JDK v1.7.0_91
        -1 unit 0m 38s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 0m 32s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        41m 56s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782589/HADOOP-12715.01.patch
        JIRA Issue HADOOP-12715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1cbe98861e52 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c07f7fa
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/branch-compile-root-jdk1.8.0_66.txt
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/branch-compile-root-jdk1.7.0_91.txt
        findbugs v3.0.0
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 30s trunk passed -1 compile 6m 2s root in trunk failed with JDK v1.8.0_66. -1 compile 0m 47s root in trunk failed with JDK v1.7.0_91. +1 checkstyle 0m 19s trunk passed +1 mvnsite 1m 8s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 2m 12s trunk passed +1 javadoc 1m 12s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 12s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 23s the patch passed -1 compile 4m 7s root in the patch failed with JDK v1.8.0_66. -1 javac 4m 7s root in the patch failed with JDK v1.8.0_66. -1 compile 0m 57s root in the patch failed with JDK v1.7.0_91. -1 javac 0m 57s root in the patch failed with JDK v1.7.0_91. +1 checkstyle 0m 22s the patch passed +1 mvnsite 1m 19s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 9s the patch passed +1 javadoc 1m 47s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 33s the patch passed with JDK v1.7.0_91 -1 unit 0m 38s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 0m 32s hadoop-common in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 41m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782589/HADOOP-12715.01.patch JIRA Issue HADOOP-12715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1cbe98861e52 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c07f7fa Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/branch-compile-root-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/branch-compile-root-jdk1.7.0_91.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8419/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Last failure seems to be YETUS-286. Reattaching...

        Show
        xiaochen Xiao Chen added a comment - Last failure seems to be YETUS-286 . Reattaching...
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 24s trunk passed
        -1 compile 2m 12s root in trunk failed with JDK v1.8.0_66.
        -1 compile 0m 47s root in trunk failed with JDK v1.7.0_91.
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 1m 8s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 2m 1s trunk passed
        +1 javadoc 0m 58s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 8s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 40s the patch passed
        -1 compile 2m 2s root in the patch failed with JDK v1.8.0_66.
        -1 javac 2m 2s root in the patch failed with JDK v1.8.0_66.
        -1 compile 2m 6s root in the patch failed with JDK v1.7.0_91.
        -1 javac 2m 6s root in the patch failed with JDK v1.7.0_91.
        +1 checkstyle 0m 17s the patch passed
        +1 mvnsite 1m 4s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 7s the patch passed
        +1 javadoc 0m 54s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 5s the patch passed with JDK v1.7.0_91
        -1 unit 0m 29s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 8m 14s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        39m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782614/HADOOP-12715.01.patch
        JIRA Issue HADOOP-12715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f4cb814d09f8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b2c155f
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/branch-compile-root-jdk1.8.0_66.txt
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/branch-compile-root-jdk1.7.0_91.txt
        findbugs v3.0.0
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 24s trunk passed -1 compile 2m 12s root in trunk failed with JDK v1.8.0_66. -1 compile 0m 47s root in trunk failed with JDK v1.7.0_91. +1 checkstyle 0m 17s trunk passed +1 mvnsite 1m 8s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 1s trunk passed +1 javadoc 0m 58s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 8s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 40s the patch passed -1 compile 2m 2s root in the patch failed with JDK v1.8.0_66. -1 javac 2m 2s root in the patch failed with JDK v1.8.0_66. -1 compile 2m 6s root in the patch failed with JDK v1.7.0_91. -1 javac 2m 6s root in the patch failed with JDK v1.7.0_91. +1 checkstyle 0m 17s the patch passed +1 mvnsite 1m 4s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 7s the patch passed +1 javadoc 0m 54s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 5s the patch passed with JDK v1.7.0_91 -1 unit 0m 29s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 8m 14s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 39m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782614/HADOOP-12715.01.patch JIRA Issue HADOOP-12715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f4cb814d09f8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b2c155f Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/branch-compile-root-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/branch-compile-root-jdk1.7.0_91.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8422/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 54s trunk passed
        +1 compile 8m 44s trunk passed with JDK v1.8.0_66
        +1 compile 7m 38s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 1m 3s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 47s trunk passed
        +1 javadoc 0m 59s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 5s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 29s the patch passed
        +1 compile 8m 2s the patch passed with JDK v1.8.0_66
        +1 javac 8m 2s the patch passed
        +1 compile 7m 16s the patch passed with JDK v1.7.0_91
        +1 javac 7m 16s the patch passed
        +1 checkstyle 0m 18s the patch passed
        +1 mvnsite 1m 12s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 0s the patch passed
        +1 javadoc 1m 0s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 8s the patch passed with JDK v1.7.0_91
        +1 unit 8m 26s hadoop-common in the patch passed with JDK v1.8.0_66.
        -1 unit 7m 45s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        70m 0s



        Reason Tests
        JDK v1.7.0_91 Failed junit tests hadoop.ipc.TestIPC
          hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782632/HADOOP-12715.01.patch
        JIRA Issue HADOOP-12715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2115091413f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2a30386
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 54s trunk passed +1 compile 8m 44s trunk passed with JDK v1.8.0_66 +1 compile 7m 38s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 15s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 59s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 5s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 29s the patch passed +1 compile 8m 2s the patch passed with JDK v1.8.0_66 +1 javac 8m 2s the patch passed +1 compile 7m 16s the patch passed with JDK v1.7.0_91 +1 javac 7m 16s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvnsite 1m 12s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 0s the patch passed +1 javadoc 1m 0s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 8s the patch passed with JDK v1.7.0_91 +1 unit 8m 26s hadoop-common in the patch passed with JDK v1.8.0_66. -1 unit 7m 45s hadoop-common in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 70m 0s Reason Tests JDK v1.7.0_91 Failed junit tests hadoop.ipc.TestIPC   hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782632/HADOOP-12715.01.patch JIRA Issue HADOOP-12715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2115091413f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2a30386 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8424/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Test failure is unrelated. (HADOOP-12605 for the TestIPC failure, would be lovely if someone can help commit. HADOOP-12588 is TestGangliaMetrics )

        Show
        xiaochen Xiao Chen added a comment - Test failure is unrelated. ( HADOOP-12605 for the TestIPC failure, would be lovely if someone can help commit. HADOOP-12588 is TestGangliaMetrics )
        Hide
        liuml07 Mingliang Liu added a comment -

        Thanks for working on this. I think your analysis is correct.

        If there is any execution exception, we swallow the exception here. Should we retry (returning false) or simply end waiting (returning true)?

        +        } catch (ExecutionException e) {
        +          LOG.error("Exception when getSize.", e);
        +          return false;
        +        }
        +      }
        
        Show
        liuml07 Mingliang Liu added a comment - Thanks for working on this. I think your analysis is correct. If there is any execution exception, we swallow the exception here. Should we retry (returning false) or simply end waiting (returning true)? + } catch (ExecutionException e) { + LOG.error( "Exception when getSize." , e); + return false ; + } + }
        Hide
        xiaochen Xiao Chen added a comment -

        Thank you for looking into this Mingliang Liu. Good point, the ExecutionException can only possibly come from LoadingCache#get, in which case I guess there's no point in retrying.

        Patch 2 is attached, the only line changed is to return true.

        Show
        xiaochen Xiao Chen added a comment - Thank you for looking into this Mingliang Liu . Good point, the ExecutionException can only possibly come from LoadingCache#get , in which case I guess there's no point in retrying. Patch 2 is attached, the only line changed is to return true.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 33s trunk passed
        +1 compile 6m 35s trunk passed with JDK v1.8.0_66
        +1 compile 7m 18s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 1m 6s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 52s trunk passed
        +1 javadoc 0m 56s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 7s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 38s the patch passed
        +1 compile 6m 32s the patch passed with JDK v1.8.0_66
        +1 javac 6m 32s the patch passed
        +1 compile 7m 21s the patch passed with JDK v1.7.0_91
        +1 javac 7m 21s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 1m 5s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 5s the patch passed
        +1 javadoc 0m 56s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 4s the patch passed with JDK v1.7.0_91
        +1 unit 8m 1s hadoop-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 0s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        66m 49s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782711/HADOOP-12715.02.patch
        JIRA Issue HADOOP-12715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 28d77aac6c49 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2a30386
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8428/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8428/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 33s trunk passed +1 compile 6m 35s trunk passed with JDK v1.8.0_66 +1 compile 7m 18s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 16s trunk passed +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 52s trunk passed +1 javadoc 0m 56s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 7s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 38s the patch passed +1 compile 6m 32s the patch passed with JDK v1.8.0_66 +1 javac 6m 32s the patch passed +1 compile 7m 21s the patch passed with JDK v1.7.0_91 +1 javac 7m 21s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 1m 5s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 5s the patch passed +1 javadoc 0m 56s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 4s the patch passed with JDK v1.7.0_91 +1 unit 8m 1s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 0s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 66m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782711/HADOOP-12715.02.patch JIRA Issue HADOOP-12715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 28d77aac6c49 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2a30386 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8428/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8428/console This message was automatically generated.
        Hide
        walter.k.su Walter Su added a comment -

        I got the same AssertionError if I delay the sync filling of getAtMost(19) manually. You're correct, thanks Xiao Chen.

        148     // Drain completely
        149     Assert.assertEquals("Failed to drain completely.", 10,
        150         vq.getAtMost("k1", 10).size());
        151     // Synchronous call
        152     Assert.assertEquals("Failed in sync call.", 10, filler.getTop().num);
        153     // Ask for more... return all
        154     GenericTestUtils.waitFor(new Supplier<Boolean>() {
            ...
        169     }, 100, 3000);
        170     Assert.assertEquals("Failed in async call.", 10, filler.getTop().num);
        171     // Drain completely after filled by the async thread
        172     Assert.assertEquals("Failed to drain completely after async.", 10,
        173         vq.getAtMost("k1", 10).size());
        

        Your comment at line 171 is good. The original comment at line 148 is misleading. When test procedure goes to line 149, the queue in LoadingCache is already empty.
        So what line 148~150 does is "trigger sync task to fill returned list, and another async task to fill the queue in the cache".
        +1 once addressed.

        Show
        walter.k.su Walter Su added a comment - I got the same AssertionError if I delay the sync filling of getAtMost(19) manually. You're correct, thanks Xiao Chen . 148 // Drain completely 149 Assert.assertEquals( "Failed to drain completely." , 10, 150 vq.getAtMost( "k1" , 10).size()); 151 // Synchronous call 152 Assert.assertEquals( "Failed in sync call." , 10, filler.getTop().num); 153 // Ask for more... return all 154 GenericTestUtils.waitFor( new Supplier< Boolean >() { ... 169 }, 100, 3000); 170 Assert.assertEquals( "Failed in async call." , 10, filler.getTop().num); 171 // Drain completely after filled by the async thread 172 Assert.assertEquals( "Failed to drain completely after async." , 10, 173 vq.getAtMost( "k1" , 10).size()); Your comment at line 171 is good. The original comment at line 148 is misleading. When test procedure goes to line 149, the queue in LoadingCache is already empty. So what line 148~150 does is "trigger sync task to fill returned list, and another async task to fill the queue in the cache". +1 once addressed.
        Hide
        xiaochen Xiao Chen added a comment -

        Thank you Walter Su for verifying, and good catch on the comment!
        Patch 3 addresses your comment. I also modified some other comments, to make the test clearer. No real code change from patch 2.

        Show
        xiaochen Xiao Chen added a comment - Thank you Walter Su for verifying, and good catch on the comment! Patch 3 addresses your comment. I also modified some other comments, to make the test clearer. No real code change from patch 2.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 0s trunk passed
        +1 compile 6m 42s trunk passed with JDK v1.8.0_66
        +1 compile 7m 37s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 1m 10s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 2m 3s trunk passed
        +1 javadoc 1m 1s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 10s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 38s the patch passed
        +1 compile 8m 38s the patch passed with JDK v1.8.0_66
        +1 javac 8m 38s the patch passed
        +1 compile 8m 6s the patch passed with JDK v1.7.0_91
        +1 javac 8m 6s the patch passed
        +1 checkstyle 0m 17s the patch passed
        +1 mvnsite 1m 8s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 19s the patch passed
        +1 javadoc 1m 2s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 14s the patch passed with JDK v1.7.0_91
        -1 unit 8m 23s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 8m 40s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 25s Patch does not generate ASF License warnings.
        71m 33s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.test.TestTimedOutTestsListener



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783988/HADOOP-12715.03.patch
        JIRA Issue HADOOP-12715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 02b38784e2f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 99829eb
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 0s trunk passed +1 compile 6m 42s trunk passed with JDK v1.8.0_66 +1 compile 7m 37s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 1m 10s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 3s trunk passed +1 javadoc 1m 1s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 10s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 38s the patch passed +1 compile 8m 38s the patch passed with JDK v1.8.0_66 +1 javac 8m 38s the patch passed +1 compile 8m 6s the patch passed with JDK v1.7.0_91 +1 javac 8m 6s the patch passed +1 checkstyle 0m 17s the patch passed +1 mvnsite 1m 8s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 19s the patch passed +1 javadoc 1m 2s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 14s the patch passed with JDK v1.7.0_91 -1 unit 8m 23s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 8m 40s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 71m 33s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.test.TestTimedOutTestsListener Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783988/HADOOP-12715.03.patch JIRA Issue HADOOP-12715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 02b38784e2f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 99829eb Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8453/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Failed test is unrelated. (Created HADOOP-12736 for that)

        Show
        xiaochen Xiao Chen added a comment - Failed test is unrelated. (Created HADOOP-12736 for that)
        Hide
        walter.k.su Walter Su added a comment -

        Committed this to trunk, branch-2, branch-2.8, branch-2.7, and branch-2.6. Thanks Xiao Chen for the contribution!

        Show
        walter.k.su Walter Su added a comment - Committed this to trunk, branch-2, branch-2.8, branch-2.7, and branch-2.6. Thanks Xiao Chen for the contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9179 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9179/)
        HADOOP-12715. TestValueQueue#testgetAtMostPolicyALL fails (waltersu4549: rev 6eacdea0e475b4fff91cedce5005a7c11749cf64)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestValueQueue.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9179 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9179/ ) HADOOP-12715 . TestValueQueue#testgetAtMostPolicyALL fails (waltersu4549: rev 6eacdea0e475b4fff91cedce5005a7c11749cf64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestValueQueue.java
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Mingliang for the review, and Walter for the review and commit!

        Show
        xiaochen Xiao Chen added a comment - Thanks Mingliang for the review, and Walter for the review and commit!
        Hide
        djp Junping Du added a comment -

        Hi, the commit patch on branch-2.6 cause build failure because getSize() is not defined in ValueQueue as HDFS-7209 is missing. I reopen this JIRA for an addendum patch to get in.

        Show
        djp Junping Du added a comment - Hi, the commit patch on branch-2.6 cause build failure because getSize() is not defined in ValueQueue as HDFS-7209 is missing. I reopen this JIRA for an addendum patch to get in.
        Hide
        djp Junping Du added a comment -

        It sounds like HDFS-7209 is part of HDFS-6891 which is not target for 2.6 branch. Given this is not critical/blocker for branch-2.6, I will revert the commit for branch-2.6 if no one objects.

        Show
        djp Junping Du added a comment - It sounds like HDFS-7209 is part of HDFS-6891 which is not target for 2.6 branch. Given this is not critical/blocker for branch-2.6, I will revert the commit for branch-2.6 if no one objects.
        Hide
        djp Junping Du added a comment -

        I just revert it from branch-2.6.

        Show
        djp Junping Du added a comment - I just revert it from branch-2.6.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks very much Junping Du for taking care of this! I think not having this in branch-2.6 should be Okay. I can also provide a branch-2.6 patch if needed.

        Show
        xiaochen Xiao Chen added a comment - Thanks very much Junping Du for taking care of this! I think not having this in branch-2.6 should be Okay. I can also provide a branch-2.6 patch if needed.
        Hide
        djp Junping Du added a comment -

        Thanks Xiao Chen. As a maintenance release, branch-2.6 is mostly for rapid release of critical/blocker bug fixes. Let's keep this so far.

        Show
        djp Junping Du added a comment - Thanks Xiao Chen . As a maintenance release, branch-2.6 is mostly for rapid release of critical/blocker bug fixes. Let's keep this so far.
        Hide
        xiaochen Xiao Chen added a comment -

        Makes sense. Thanks again Junping.

        Show
        xiaochen Xiao Chen added a comment - Makes sense. Thanks again Junping.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Closing the JIRA as part of 2.7.3 release.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

          People

          • Assignee:
            xiaochen Xiao Chen
            Reporter:
            xiaochen Xiao Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development