Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12605

Fix intermittent failure of TestIPC.testIpcWithReaderQueuing

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Target Version/s:
    1. HADOOP-12605.001.patch
      4 kB
      Masatake Iwasaki
    2. HADOOP-12605.002.patch
      3 kB
      Masatake Iwasaki
    3. HADOOP-12605.003.patch
      2 kB
      Masatake Iwasaki
    4. HADOOP-12605.004.patch
      2 kB
      Masatake Iwasaki
    5. HADOOP-12605.005.patch
      3 kB
      Xiao Chen

      Issue Links

        Activity

        Hide
        eepayne Eric Payne added a comment -

        Thanks Masatake Iwasaki for this fix.

        I backported it to branch-2.8.

        Show
        eepayne Eric Payne added a comment - Thanks Masatake Iwasaki for this fix. I backported it to branch-2.8.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Committed this to trunk and branch-2. Thanks, Xiao Chen and Steve Loughran.

        Show
        iwasakims Masatake Iwasaki added a comment - Committed this to trunk and branch-2. Thanks, Xiao Chen and Steve Loughran .
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9146 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9146/)
        HADOOP-12605. Fix intermittent failure of (iwasakims: rev 9eec6cbedcc53e6dd306a4a578a22937ae239260)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9146 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9146/ ) HADOOP-12605 . Fix intermittent failure of (iwasakims: rev 9eec6cbedcc53e6dd306a4a578a22937ae239260) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks for the +1 Steve Loughran. Would you have time to commit this soon? Thanks!

        Show
        xiaochen Xiao Chen added a comment - Thanks for the +1 Steve Loughran . Would you have time to commit this soon? Thanks!
        Hide
        stevel@apache.org Steve Loughran added a comment -

        +1

        Show
        stevel@apache.org Steve Loughran added a comment - +1
        Hide
        xiaochen Xiao Chen added a comment -

        Hi mighty watchers, could some of you please review? And happy holidays all!

        Show
        xiaochen Xiao Chen added a comment - Hi mighty watchers, could some of you please review? And happy holidays all!
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Thanks, Xiao Chen!

        Show
        iwasakims Masatake Iwasaki added a comment - Thanks, Xiao Chen !
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 37s trunk passed
        +1 compile 8m 9s trunk passed with JDK v1.8.0_66
        +1 compile 8m 48s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 1m 3s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 47s trunk passed
        +1 javadoc 0m 54s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 1s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 38s the patch passed
        +1 compile 8m 18s the patch passed with JDK v1.8.0_66
        +1 javac 8m 18s the patch passed
        +1 compile 8m 44s the patch passed with JDK v1.7.0_91
        +1 javac 8m 44s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 1m 0s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 57s the patch passed
        +1 javadoc 0m 50s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 3s the patch passed with JDK v1.7.0_91
        +1 unit 6m 46s hadoop-common in the patch passed with JDK v1.8.0_66.
        +1 unit 7m 6s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        69m 3s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778528/HADOOP-12605.005.patch
        JIRA Issue HADOOP-12605
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 96fde8167777 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e63388f
        findbugs v3.0.0
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8277/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8277/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 37s trunk passed +1 compile 8m 9s trunk passed with JDK v1.8.0_66 +1 compile 8m 48s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 15s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 54s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 1s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 38s the patch passed +1 compile 8m 18s the patch passed with JDK v1.8.0_66 +1 javac 8m 18s the patch passed +1 compile 8m 44s the patch passed with JDK v1.7.0_91 +1 javac 8m 44s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 50s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_91 +1 unit 6m 46s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 7m 6s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 69m 3s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778528/HADOOP-12605.005.patch JIRA Issue HADOOP-12605 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 96fde8167777 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e63388f findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8277/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8277/console This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Masatake Iwasaki for the new rev! LGTM.
        I went ahead and fixed the whitespace/javac warnings, hope you don't mind.

        Show
        xiaochen Xiao Chen added a comment - Thanks Masatake Iwasaki for the new rev! LGTM. I went ahead and fixed the whitespace/javac warnings, hope you don't mind.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 56s trunk passed
        +1 compile 9m 26s trunk passed with JDK v1.8.0_66
        +1 compile 9m 40s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 1m 7s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 2m 1s trunk passed
        +1 javadoc 0m 56s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 9s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 42s the patch passed
        +1 compile 9m 30s the patch passed with JDK v1.8.0_66
        -1 javac 15m 18s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 729, now 729).
        +1 javac 9m 30s the patch passed
        +1 compile 10m 3s the patch passed with JDK v1.7.0_91
        -1 javac 25m 21s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 723, now 723).
        +1 javac 10m 3s the patch passed
        +1 checkstyle 0m 17s the patch passed
        +1 mvnsite 1m 4s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 7s the patch passed
        +1 javadoc 0m 59s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 10s the patch passed with JDK v1.7.0_91
        -1 unit 7m 9s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 7m 45s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        77m 20s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778475/HADOOP-12605.004.patch
        JIRA Issue HADOOP-12605
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 258fdde9b615 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bfadf11
        findbugs v3.0.0
        javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt
        javac root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_91.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/whitespace-eol.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 56s trunk passed +1 compile 9m 26s trunk passed with JDK v1.8.0_66 +1 compile 9m 40s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 1m 7s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 2m 1s trunk passed +1 javadoc 0m 56s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 9s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 42s the patch passed +1 compile 9m 30s the patch passed with JDK v1.8.0_66 -1 javac 15m 18s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 729, now 729). +1 javac 9m 30s the patch passed +1 compile 10m 3s the patch passed with JDK v1.7.0_91 -1 javac 25m 21s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 723, now 723). +1 javac 10m 3s the patch passed +1 checkstyle 0m 17s the patch passed +1 mvnsite 1m 4s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 7s the patch passed +1 javadoc 0m 59s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 10s the patch passed with JDK v1.7.0_91 -1 unit 7m 9s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 7m 45s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 77m 20s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.TestLocalFsFCStatistics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778475/HADOOP-12605.004.patch JIRA Issue HADOOP-12605 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 258fdde9b615 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bfadf11 findbugs v3.0.0 javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt javac root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_91.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8276/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 29s trunk passed
        +1 compile 9m 34s trunk passed with JDK v1.8.0_66
        +1 compile 9m 35s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 1m 11s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 2m 0s trunk passed
        +1 javadoc 1m 0s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 8s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 42s the patch passed
        +1 compile 8m 43s the patch passed with JDK v1.8.0_66
        -1 javac 14m 34s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 730, now 730).
        +1 javac 8m 43s the patch passed
        +1 compile 9m 32s the patch passed with JDK v1.7.0_91
        -1 javac 24m 6s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 723, now 723).
        +1 javac 9m 32s the patch passed
        +1 checkstyle 0m 20s the patch passed
        +1 mvnsite 1m 18s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 findbugs 2m 7s the patch passed
        +1 javadoc 0m 58s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 10s the patch passed with JDK v1.7.0_91
        +1 unit 8m 32s hadoop-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 25s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        77m 55s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778468/HADOOP-12605.003.patch
        JIRA Issue HADOOP-12605
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f7d876fdbb36 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bfadf11
        findbugs v3.0.0
        javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt
        javac root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_91.txt
        whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/artifact/patchprocess/whitespace-eol.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 29s trunk passed +1 compile 9m 34s trunk passed with JDK v1.8.0_66 +1 compile 9m 35s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 1m 11s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 2m 0s trunk passed +1 javadoc 1m 0s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 8s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 42s the patch passed +1 compile 8m 43s the patch passed with JDK v1.8.0_66 -1 javac 14m 34s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 730, now 730). +1 javac 8m 43s the patch passed +1 compile 9m 32s the patch passed with JDK v1.7.0_91 -1 javac 24m 6s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 1 new issues (was 723, now 723). +1 javac 9m 32s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 1m 18s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 2m 7s the patch passed +1 javadoc 0m 58s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 10s the patch passed with JDK v1.7.0_91 +1 unit 8m 32s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 25s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 77m 55s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778468/HADOOP-12605.003.patch JIRA Issue HADOOP-12605 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f7d876fdbb36 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bfadf11 findbugs v3.0.0 javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt javac root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_91.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/artifact/patchprocess/whitespace-eol.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8275/console This message was automatically generated.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Oops. We should have the verify for all initial clients. I updated the patch.

        Show
        iwasakims Masatake Iwasaki added a comment - Oops. We should have the verify for all initial clients. I updated the patch.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Thanks Xiao Chen, I attached 003.

        • using Whitebox instead of changing visibility of field.
        • updated comment in the test.
        Show
        iwasakims Masatake Iwasaki added a comment - Thanks Xiao Chen , I attached 003. using Whitebox instead of changing visibility of field. updated comment in the test.
        Hide
        xiaochen Xiao Chen added a comment -

        There is a chance for additional connections to be taken out of pendingConnections before reader blocks on the second message from the clients.

        Makes sense to me. I didn't catch this part. Strictly speaking, this is a flaw in the reader, rather than a flaky test. But IMO this is fine and we can just update the test to make sure we don't run into it. Also thanks for the debugging log - I tried locally to run thousands of times, but never had it fail!

        The new approach looks better. Making the callQueue to be protected brings up a new check style warning though, not sure what's the best practice here. Another minor comment is to update the comment in test after " // let subsequent readers jam the callq, will happen immediately ". I've tested locally that patch 2 passes. Thanks a lot for the work and analysis, Masatake Iwasaki.

        Show
        xiaochen Xiao Chen added a comment - There is a chance for additional connections to be taken out of pendingConnections before reader blocks on the second message from the clients. Makes sense to me. I didn't catch this part. Strictly speaking, this is a flaw in the reader, rather than a flaky test. But IMO this is fine and we can just update the test to make sure we don't run into it. Also thanks for the debugging log - I tried locally to run thousands of times, but never had it fail! The new approach looks better. Making the callQueue to be protected brings up a new check style warning though, not sure what's the best practice here. Another minor comment is to update the comment in test after " // let subsequent readers jam the callq, will happen immediately ". I've tested locally that patch 2 passes. Thanks a lot for the work and analysis, Masatake Iwasaki .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 9m 39s trunk passed
        +1 compile 13m 35s trunk passed with JDK v1.8.0_66
        +1 compile 11m 53s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 20s trunk passed
        +1 mvnsite 1m 22s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        +1 findbugs 2m 23s trunk passed
        +1 javadoc 1m 22s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 25s trunk passed with JDK v1.7.0_91
        +1 mvninstall 1m 56s the patch passed
        +1 compile 13m 31s the patch passed with JDK v1.8.0_66
        +1 javac 13m 31s the patch passed
        +1 compile 11m 45s the patch passed with JDK v1.7.0_91
        +1 javac 11m 45s the patch passed
        -1 checkstyle 0m 20s Patch generated 1 new checkstyle issues in hadoop-common-project/hadoop-common (total was 215, now 216).
        +1 mvnsite 1m 18s the patch passed
        +1 mvneclipse 0m 16s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 34s the patch passed
        +1 javadoc 1m 17s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 24s the patch passed with JDK v1.7.0_91
        -1 unit 9m 12s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 8m 59s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 28s Patch does not generate ASF License warnings.
        96m 31s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.find.TestFind
          hadoop.test.TestTimedOutTestsListener
        JDK v1.7.0_91 Failed junit tests hadoop.ha.TestZKFailoverController
          hadoop.ipc.TestRPC



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778410/HADOOP-12605.002.patch
        JIRA Issue HADOOP-12605
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e88837122603 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bd5e207
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 39s trunk passed +1 compile 13m 35s trunk passed with JDK v1.8.0_66 +1 compile 11m 53s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 20s trunk passed +1 mvnsite 1m 22s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 2m 23s trunk passed +1 javadoc 1m 22s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 25s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 56s the patch passed +1 compile 13m 31s the patch passed with JDK v1.8.0_66 +1 javac 13m 31s the patch passed +1 compile 11m 45s the patch passed with JDK v1.7.0_91 +1 javac 11m 45s the patch passed -1 checkstyle 0m 20s Patch generated 1 new checkstyle issues in hadoop-common-project/hadoop-common (total was 215, now 216). +1 mvnsite 1m 18s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 34s the patch passed +1 javadoc 1m 17s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 24s the patch passed with JDK v1.7.0_91 -1 unit 9m 12s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 8m 59s hadoop-common in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 28s Patch does not generate ASF License warnings. 96m 31s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.find.TestFind   hadoop.test.TestTimedOutTestsListener JDK v1.7.0_91 Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.ipc.TestRPC Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778410/HADOOP-12605.002.patch JIRA Issue HADOOP-12605 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e88837122603 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bd5e207 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8271/console This message was automatically generated.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        I changed the approach on attached 002. It makes test deterministic by making sure that readers are blocking on callQueue by spying callQueue.

        Show
        iwasakims Masatake Iwasaki added a comment - I changed the approach on attached 002. It makes test deterministic by making sure that readers are blocking on callQueue by spying callQueue.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        I'm going to look again the patch and update it.

        Show
        iwasakims Masatake Iwasaki added a comment - I'm going to look again the patch and update it.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Here is the DEBUG log of failed checkBlocking(1, 1, 1).

        2015-12-14 18:09:56,893 INFO  ipc.CallQueueManager (CallQueueManager.java:<init>(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue
        2015-12-14 18:09:56,894 INFO  ipc.Server (Server.java:run(727)) - Starting Socket Reader #1 for port 50977
        2015-12-14 18:09:56,897 INFO  ipc.Server (Server.java:run(957)) - IPC Server Responder: starting
        2015-12-14 18:09:56,897 INFO  ipc.Server (Server.java:run(797)) - IPC Server listener on 50977: starting
        2015-12-14 18:09:56,897 DEBUG ipc.Server (Server.java:run(2262)) - IPC Server handler 0 on 50977: starting
        2015-12-14 18:09:56,904 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34212; # active connections: 1; # queued calls: 0
        2015-12-14 18:09:56,906 DEBUG ipc.Server (Server.java:processOneRpc(1979)) -  got #-3
        2015-12-14 18:09:56,963 DEBUG ipc.Server (Server.java:authorizeConnection(2183)) - Successfully authorized userInfo {
        }
        
        2015-12-14 18:09:56,963 DEBUG ipc.Server (Server.java:processOneRpc(1979)) -  got #6
        2015-12-14 18:09:56,963 DEBUG ipc.Server (Server.java:run(2271)) - IPC Server handler 0 on 50977: 46 from 127.0.0.1:34212 Call#6 Retry#0 for RpcKind RPC_BUILTIN
        2015-12-14 18:09:56,964 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34213; # active connections: 2; # queued calls: 0
        2015-12-14 18:09:56,965 DEBUG ipc.Server (Server.java:processOneRpc(1979)) -  got #-3
        2015-12-14 18:09:56,965 DEBUG ipc.Server (Server.java:authorizeConnection(2183)) - Successfully authorized userInfo {
        }
        
        2015-12-14 18:09:56,965 DEBUG ipc.Server (Server.java:processOneRpc(1979)) -  got #7
        2015-12-14 18:09:56,967 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34214; # active connections: 3; # queued calls: 1
        2015-12-14 18:09:56,967 INFO  ipc.TestIPC (TestIPC.java:checkBlocking(779)) - (initial clients) need:3 connections have:3
        2015-12-14 18:09:56,968 INFO  ipc.TestIPC (TestIPC.java:checkBlocking(781)) - ipc layer should be blocked
        2015-12-14 18:09:56,969 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34215; # active connections: 4; # queued calls: 1
        2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:processOneRpc(1979)) -  got #-3
        2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:authorizeConnection(2183)) - Successfully authorized userInfo {
        }
        
        2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:processOneRpc(1979)) -  got #8
        2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34216; # active connections: 5; # queued calls: 1
        2015-12-14 18:09:56,971 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34217; # active connections: 6; # queued calls: 1
        2015-12-14 18:09:56,980 INFO  ipc.TestIPC (TestIPC.java:checkBlocking(801)) - (max clients) need:5 connections have:6
        
        Show
        iwasakims Masatake Iwasaki added a comment - Here is the DEBUG log of failed checkBlocking(1, 1, 1) . 2015-12-14 18:09:56,893 INFO ipc.CallQueueManager (CallQueueManager.java:<init>(60)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2015-12-14 18:09:56,894 INFO ipc.Server (Server.java:run(727)) - Starting Socket Reader #1 for port 50977 2015-12-14 18:09:56,897 INFO ipc.Server (Server.java:run(957)) - IPC Server Responder: starting 2015-12-14 18:09:56,897 INFO ipc.Server (Server.java:run(797)) - IPC Server listener on 50977: starting 2015-12-14 18:09:56,897 DEBUG ipc.Server (Server.java:run(2262)) - IPC Server handler 0 on 50977: starting 2015-12-14 18:09:56,904 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34212; # active connections: 1; # queued calls: 0 2015-12-14 18:09:56,906 DEBUG ipc.Server (Server.java:processOneRpc(1979)) - got #-3 2015-12-14 18:09:56,963 DEBUG ipc.Server (Server.java:authorizeConnection(2183)) - Successfully authorized userInfo { } 2015-12-14 18:09:56,963 DEBUG ipc.Server (Server.java:processOneRpc(1979)) - got #6 2015-12-14 18:09:56,963 DEBUG ipc.Server (Server.java:run(2271)) - IPC Server handler 0 on 50977: 46 from 127.0.0.1:34212 Call#6 Retry#0 for RpcKind RPC_BUILTIN 2015-12-14 18:09:56,964 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34213; # active connections: 2; # queued calls: 0 2015-12-14 18:09:56,965 DEBUG ipc.Server (Server.java:processOneRpc(1979)) - got #-3 2015-12-14 18:09:56,965 DEBUG ipc.Server (Server.java:authorizeConnection(2183)) - Successfully authorized userInfo { } 2015-12-14 18:09:56,965 DEBUG ipc.Server (Server.java:processOneRpc(1979)) - got #7 2015-12-14 18:09:56,967 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34214; # active connections: 3; # queued calls: 1 2015-12-14 18:09:56,967 INFO ipc.TestIPC (TestIPC.java:checkBlocking(779)) - (initial clients) need:3 connections have:3 2015-12-14 18:09:56,968 INFO ipc.TestIPC (TestIPC.java:checkBlocking(781)) - ipc layer should be blocked 2015-12-14 18:09:56,969 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34215; # active connections: 4; # queued calls: 1 2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:processOneRpc(1979)) - got #-3 2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:authorizeConnection(2183)) - Successfully authorized userInfo { } 2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:processOneRpc(1979)) - got #8 2015-12-14 18:09:56,970 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34216; # active connections: 5; # queued calls: 1 2015-12-14 18:09:56,971 DEBUG ipc.Server (Server.java:register(2969)) - Server connection from 127.0.0.1:34217; # active connections: 6; # queued calls: 1 2015-12-14 18:09:56,980 INFO ipc.TestIPC (TestIPC.java:checkBlocking(801)) - (max clients) need:5 connections have:6
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Also, the failure happens for checkBlocking(1, 1, 1), so I'll discuss this case here for simplicity. With the firstCallLatch in place in the test, I didn't find anywhere the server can accept more connections.

        Newly created Client sends client context whose call id is -3 at first. Given the first message is processed by Connection#processRpcOutOfBandRequest which does not use callQueue, reader thread taking the connection out of pendingConnections and processing the connection context (in Reader#doRunLoop) will not block on callQueue even if callQueue is filled. There is a chance for additional connections to be taken out of pendingConnections before reader blocks on the second message from the clients.

                if (callId < 0) { // callIds typically used during connection setup                                                                                                                                                                                
                  processRpcOutOfBandRequest(header, dis);
                } else if (!connectionContextRead) {
                  throw new WrappedRpcServerException(
                      RpcErrorCodeProto.FATAL_INVALID_RPC_HEADER,
                      "Connection context not established");
                } else {
                  processRpcRequest(header, dis);
                }
        
        Show
        iwasakims Masatake Iwasaki added a comment - Also, the failure happens for checkBlocking(1, 1, 1), so I'll discuss this case here for simplicity. With the firstCallLatch in place in the test, I didn't find anywhere the server can accept more connections. Newly created Client sends client context whose call id is -3 at first. Given the first message is processed by Connection#processRpcOutOfBandRequest which does not use callQueue , reader thread taking the connection out of pendingConnections and processing the connection context (in Reader#doRunLoop ) will not block on callQueue even if callQueue is filled. There is a chance for additional connections to be taken out of pendingConnections before reader blocks on the second message from the clients. if (callId < 0) { // callIds typically used during connection setup processRpcOutOfBandRequest(header, dis); } else if (!connectionContextRead) { throw new WrappedRpcServerException( RpcErrorCodeProto.FATAL_INVALID_RPC_HEADER, "Connection context not established" ); } else { processRpcRequest(header, dis); }
        Hide
        xiaochen Xiao Chen added a comment -

        Hi Masatake Iwasaki,
        Thanks for the explanation and the patch. Sorry for my delayed response.

        I'm not sure about relaxing the check between server.getNumOpenConnections() and maxAccept - that would no longer check the intention of 'we didn't go over'. I hope the original author Daryn Sharp could take a look and advice.

        Also, the failure happens for checkBlocking(1, 1, 1), so I'll discuss this case here for simplicity. With the firstCallLatch in place in the test, I didn't find anywhere the server can accept more connections. Could you further explain your theory?
        From my understanding, the call goes briefly like this: client call -> server's listener -> add to readerQ -> reader take from readerQ -> put to callQ -> handler take from callQ -> call (blocked by the latch). In the (1, 1, 1) case we only have 1 reader with readerQ size=1, and callQ size =1, so IMO neither the round robin nor the loop of taking pendingConnections should contribute anything.

        I'm trying to hack the test to result in a similar failure but haven't succeeded so far. Masatake Iwasaki, would you please share some reproduce code if possible? Thanks very much.

        Show
        xiaochen Xiao Chen added a comment - Hi Masatake Iwasaki , Thanks for the explanation and the patch. Sorry for my delayed response. I'm not sure about relaxing the check between server.getNumOpenConnections() and maxAccept - that would no longer check the intention of 'we didn't go over'. I hope the original author Daryn Sharp could take a look and advice. Also, the failure happens for checkBlocking(1, 1, 1) , so I'll discuss this case here for simplicity. With the firstCallLatch in place in the test, I didn't find anywhere the server can accept more connections. Could you further explain your theory? From my understanding, the call goes briefly like this: client call -> server's listener -> add to readerQ -> reader take from readerQ -> put to callQ -> handler take from callQ -> call (blocked by the latch). In the (1, 1, 1) case we only have 1 reader with readerQ size=1, and callQ size =1, so IMO neither the round robin nor the loop of taking pendingConnections should contribute anything. I'm trying to hack the test to result in a similar failure but haven't succeeded so far. Masatake Iwasaki , would you please share some reproduce code if possible? Thanks very much.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 9m 26s trunk passed
        +1 compile 11m 34s trunk passed with JDK v1.8.0_66
        +1 compile 11m 15s trunk passed with JDK v1.7.0_85
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 1m 25s trunk passed
        +1 mvneclipse 0m 18s trunk passed
        +1 findbugs 2m 26s trunk passed
        +1 javadoc 1m 17s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 28s trunk passed with JDK v1.7.0_85
        +1 mvninstall 2m 5s the patch passed
        +1 compile 12m 45s the patch passed with JDK v1.8.0_66
        +1 javac 12m 45s the patch passed
        +1 compile 12m 16s the patch passed with JDK v1.7.0_85
        +1 javac 12m 16s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 1m 29s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 46s the patch passed
        +1 javadoc 1m 15s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 25s the patch passed with JDK v1.7.0_85
        -1 unit 10m 21s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 9m 52s hadoop-common in the patch passed with JDK v1.7.0_85.
        +1 asflicense 0m 43s Patch does not generate ASF License warnings.
        96m 47s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.find.TestIname
          hadoop.ipc.TestDecayRpcScheduler
          hadoop.metrics2.impl.TestGangliaMetrics
          hadoop.crypto.key.TestCachingKeyProvider
          hadoop.test.TestTimedOutTestsListener
          hadoop.fs.shell.find.TestFind



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12774952/HADOOP-12605.001.patch
        JIRA Issue HADOOP-12605
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bceec109ee23 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1c05393
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 76MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 26s trunk passed +1 compile 11m 34s trunk passed with JDK v1.8.0_66 +1 compile 11m 15s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 22s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 2m 26s trunk passed +1 javadoc 1m 17s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 28s trunk passed with JDK v1.7.0_85 +1 mvninstall 2m 5s the patch passed +1 compile 12m 45s the patch passed with JDK v1.8.0_66 +1 javac 12m 45s the patch passed +1 compile 12m 16s the patch passed with JDK v1.7.0_85 +1 javac 12m 16s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 1m 29s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 46s the patch passed +1 javadoc 1m 15s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 25s the patch passed with JDK v1.7.0_85 -1 unit 10m 21s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 9m 52s hadoop-common in the patch passed with JDK v1.7.0_85. +1 asflicense 0m 43s Patch does not generate ASF License warnings. 96m 47s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.find.TestIname   hadoop.ipc.TestDecayRpcScheduler   hadoop.metrics2.impl.TestGangliaMetrics   hadoop.crypto.key.TestCachingKeyProvider   hadoop.test.TestTimedOutTestsListener   hadoop.fs.shell.find.TestFind Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12774952/HADOOP-12605.001.patch JIRA Issue HADOOP-12605 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bceec109ee23 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1c05393 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8169/console This message was automatically generated.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        I attached 001.

        checkBlocking now starts callQ + handlers clients and waiting until callQueue is filled at first. This is needed to avoid situation as server.getNumOpenConnections() < maxAccept.

        Then the test starts all clients and waiting until server.getNumOpenConnections() >= maxAccept stands. More than maxAccept connections could be opened before readers are blocked but it is not problem.

        Show
        iwasakims Masatake Iwasaki added a comment - I attached 001. checkBlocking now starts callQ + handlers clients and waiting until callQueue is filled at first. This is needed to avoid situation as server.getNumOpenConnections() < maxAccept . Then the test starts all clients and waiting until server.getNumOpenConnections() >= maxAccept stands. More than maxAccept connections could be opened before readers are blocked but it is not problem.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        The 2nd comment (the maxAccept < server.getNumOpenConnections() case), I could be wrong, but in test we block all the calls so this shouldn't happen in the test case, right?

        The problem is that it is difficult for waiting until all readers are blocked on callQueue. Waiting for server.getNumOpenConnections() >= initialClients is not enough. We need to expose internal of Server for checking reader is blocked. I think we don't need to make sure that assertEquals(maxAccept, server.getNumOpenConnections()); because it is not contract provided by Server. Just assertTrue(server.getNumOpenConnections() >= maxAccept); is fine.

        The 3rd comment (the maxAccept < server.getNumOpenConnections() in checkBlocking(1, 1, 1) case), since in Connection#readAndProcess we have the following code, so even for new client connections 1 rpc call should be processed too. I confirmed this by some console printing.

        You are right. The problem about assertEquals(maxAccept, server.getNumOpenConnections()); just holds for checkBlocking(1, 1, 1) too.

        Show
        iwasakims Masatake Iwasaki added a comment - The 2nd comment (the maxAccept < server.getNumOpenConnections() case), I could be wrong, but in test we block all the calls so this shouldn't happen in the test case, right? The problem is that it is difficult for waiting until all readers are blocked on callQueue. Waiting for server.getNumOpenConnections() >= initialClients is not enough. We need to expose internal of Server for checking reader is blocked. I think we don't need to make sure that assertEquals(maxAccept, server.getNumOpenConnections()); because it is not contract provided by Server . Just assertTrue(server.getNumOpenConnections() >= maxAccept); is fine. The 3rd comment (the maxAccept < server.getNumOpenConnections() in checkBlocking(1, 1, 1) case), since in Connection#readAndProcess we have the following code, so even for new client connections 1 rpc call should be processed too. I confirmed this by some console printing. You are right. The problem about assertEquals(maxAccept, server.getNumOpenConnections()); just holds for checkBlocking(1, 1, 1) too.
        Hide
        xiaochen Xiao Chen added a comment -

        Hi Masatake Iwasaki,
        Thanks for looking into this issue! It's great that you looked into the implementation of the Server class. - I stopped after looking HADOOP-9956 when I tried to fix HADOOP-10406 and thought it's just a test failure. Thanks for inspiring me to look into the implementation, and good RCA overall.

        Your 1st comment (the maxAccept > server.getNumOpenConnections() case) makes sense to me.
        The 2nd comment (the maxAccept < server.getNumOpenConnections() case), I could be wrong, but in test we block all the calls so this shouldn't happen in the test case, right?
        The 3rd comment (the maxAccept < server.getNumOpenConnections() in checkBlocking(1, 1, 1) case), since in Connection#readAndProcess we have the following code, so even for new client connections 1 rpc call should be processed too. I confirmed this by some console printing.

         boolean isHeaderRead = connectionContextRead; 

        I guess you're probably already working on it - I think it'd be great if we can manage to 100% reproduce the issue first.

        Show
        xiaochen Xiao Chen added a comment - Hi Masatake Iwasaki , Thanks for looking into this issue! It's great that you looked into the implementation of the Server class. - I stopped after looking HADOOP-9956 when I tried to fix HADOOP-10406 and thought it's just a test failure. Thanks for inspiring me to look into the implementation, and good RCA overall. Your 1st comment (the maxAccept > server.getNumOpenConnections() case) makes sense to me. The 2nd comment (the maxAccept < server.getNumOpenConnections() case), I could be wrong, but in test we block all the calls so this shouldn't happen in the test case, right? The 3rd comment (the maxAccept < server.getNumOpenConnections() in checkBlocking(1, 1, 1) case), since in Connection#readAndProcess we have the following code, so even for new client connections 1 rpc call should be processed too. I confirmed this by some console printing. boolean isHeaderRead = connectionContextRead; I guess you're probably already working on it - I think it'd be great if we can manage to 100% reproduce the issue first.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        The reader processes connection context for newly created client connection by calling processRpcOutOfBandRequest. This does not result in calling TestServerQueue#call which blocks. Listener could put multiple connections before Handler is blocked and callQueue is filled. It could be maxAccept < server.getNumOpenConnections() even in checkBlocking(1, 1, 1) due to this.

        Show
        iwasakims Masatake Iwasaki added a comment - The reader processes connection context for newly created client connection by calling processRpcOutOfBandRequest . This does not result in calling TestServerQueue#call which blocks. Listener could put multiple connections before Handler is blocked and callQueue is filled. It could be maxAccept < server.getNumOpenConnections() even in checkBlocking(1, 1, 1) due to this.
        Hide
        iwasakims Masatake Iwasaki added a comment -
              private synchronized void doRunLoop() {
                while (running) {
                  SelectionKey key = null;
                  try {
                    // consume as many connections as currently queued to avoid
                    // unbridled acceptance of connections that starves the select
                    int size = pendingConnections.size();
                    for (int i=size; i>0; i--) {
                      Connection conn = pendingConnections.take();
                      conn.channel.register(readSelector, SelectionKey.OP_READ, conn);
                    }
        

        The calculation of maxAccept = initialClients + readers*readerQ + 1 assumes that 1 connection is taken out from queue at a time but reader take all pending connections out from queue in each loop in Reader#doRunLoop. The number of connections accepted until Listner blocks could be greater than the maxAccept if Listner put multiple connections to reader's queue before reader process it. (maxAccept < server.getNumOpenConnections())

        Show
        iwasakims Masatake Iwasaki added a comment - private synchronized void doRunLoop() { while (running) { SelectionKey key = null ; try { // consume as many connections as currently queued to avoid // unbridled acceptance of connections that starves the select int size = pendingConnections.size(); for ( int i=size; i>0; i--) { Connection conn = pendingConnections.take(); conn.channel.register(readSelector, SelectionKey.OP_READ, conn); } The calculation of maxAccept = initialClients + readers*readerQ + 1 assumes that 1 connection is taken out from queue at a time but reader take all pending connections out from queue in each loop in Reader#doRunLoop . The number of connections accepted until Listner blocks could be greater than the maxAccept if Listner put multiple connections to reader's queue before reader process it. ( maxAccept < server.getNumOpenConnections() )
        Hide
        iwasakims Masatake Iwasaki added a comment -
            final int initialClients = readers + callQ + handlers;
            // max connections we should ever end up accepting at once
            final int maxAccept = initialClients + readers*readerQ + 1; // 1 = listener
        

        The test expects that all readers' queue (pendingConnections) is full when Listener is blocked but the assumption is wrong. Listener put connections to queue (by Reader#addConnection) in round-robin along with Listener#getReader. Queued connections are processed in non-deterministic order by reader threads. If one of the reader's queue becomes full first then Listener trying to add connection to the queue will block even if other reader's queue have room. The number of connections accepted until Listener blocks could be less than the maxAccept due to this. (maxAccept > server.getNumOpenConnections())

        Show
        iwasakims Masatake Iwasaki added a comment - final int initialClients = readers + callQ + handlers; // max connections we should ever end up accepting at once final int maxAccept = initialClients + readers*readerQ + 1; // 1 = listener The test expects that all readers' queue ( pendingConnections ) is full when Listener is blocked but the assumption is wrong. Listener put connections to queue (by Reader#addConnection ) in round-robin along with Listener#getReader . Queued connections are processed in non-deterministic order by reader threads. If one of the reader's queue becomes full first then Listener trying to add connection to the queue will block even if other reader's queue have room. The number of connections accepted until Listener blocks could be less than the maxAccept due to this. ( maxAccept > server.getNumOpenConnections() )
        Hide
        iwasakims Masatake Iwasaki added a comment -

        I rethought about this. It would be better to do follow-up work here because the patch of HADOOP-10406 was already committed. The patch was improvement but just turned out to be not enough.

        Show
        iwasakims Masatake Iwasaki added a comment - I rethought about this. It would be better to do follow-up work here because the patch of HADOOP-10406 was already committed. The patch was improvement but just turned out to be not enough.
        Hide
        iwasakims Masatake Iwasaki added a comment -

        Thanks, Xiao Chen.

        Show
        iwasakims Masatake Iwasaki added a comment - Thanks, Xiao Chen .
        Hide
        xiaochen Xiao Chen added a comment -

        sorry typo, *please feel free to take that JIRA.

        Show
        xiaochen Xiao Chen added a comment - sorry typo, *please feel free to take that JIRA.
        Hide
        xiaochen Xiao Chen added a comment -

        This looks to be the same as the reopened failure in HADOOP-10406. I haven't got a chance to work on it. Masatake Iwasaki feel free to it if you got time.

        Show
        xiaochen Xiao Chen added a comment - This looks to be the same as the reopened failure in HADOOP-10406 . I haven't got a chance to work on it. Masatake Iwasaki feel free to it if you got time.
        Hide
        iwasakims Masatake Iwasaki added a comment -
        java.lang.AssertionError: expected:<5> but was:<10>
        	at org.junit.Assert.fail(Assert.java:88)
        	at org.junit.Assert.failNotEquals(Assert.java:743)
        	at org.junit.Assert.assertEquals(Assert.java:118)
        	at org.junit.Assert.assertEquals(Assert.java:555)
        	at org.junit.Assert.assertEquals(Assert.java:542)
        	at org.apache.hadoop.ipc.TestIPC.checkBlocking(TestIPC.java:804)
        	at org.apache.hadoop.ipc.TestIPC.testIpcWithReaderQueuing(TestIPC.java:695)
        
        Show
        iwasakims Masatake Iwasaki added a comment - java.lang.AssertionError: expected:<5> but was:<10> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.hadoop.ipc.TestIPC.checkBlocking(TestIPC.java:804) at org.apache.hadoop.ipc.TestIPC.testIpcWithReaderQueuing(TestIPC.java:695)

          People

          • Assignee:
            iwasakims Masatake Iwasaki
            Reporter:
            iwasakims Masatake Iwasaki
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development