Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: namenode, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The test TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. The stack infos:

      org.junit.ComparisonFailure: expected:<[old gs data
      new gs data
      ]> but was:<[]>
      	at org.junit.Assert.assertEquals(Assert.java:115)
      	at org.junit.Assert.assertEquals(Assert.java:144)
      	at org.apache.hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation.testRWRInvalidation(TestRBWBlockInvalidation.java:225)
      

      The issue is caused by the blocks reported not completed, similar to HDFS-10499.

      1. HDFS-11105.002.patch
        4 kB
        Yiqun Lin
      2. HDFS-11105.001.patch
        3 kB
        Yiqun Lin

        Activity

        Hide
        linyiqun Yiqun Lin added a comment -

        Thanks Mingliang Liu for the review and commit.

        Show
        linyiqun Yiqun Lin added a comment - Thanks Mingliang Liu for the review and commit.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10847 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10847/)
        HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails (liuml07: rev c90891e7b3afaa66dbca967b83a9997e80e893d8)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10847 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10847/ ) HDFS-11105 . TestRBWBlockInvalidation#testRWRInvalidation fails (liuml07: rev c90891e7b3afaa66dbca967b83a9997e80e893d8) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
        Hide
        liuml07 Mingliang Liu added a comment -

        Committed to trunk through branch-2.8 branches. Thanks for your contribution, Yiqun Lin.

        Show
        liuml07 Mingliang Liu added a comment - Committed to trunk through branch-2.8 branches. Thanks for your contribution, Yiqun Lin .
        Hide
        liuml07 Mingliang Liu added a comment -

        I think the patch makes sense; we need to make sure the condition is met before we do dependent operations and/or assertions.

        +1

        I'll hold on commit this in 2 days in case someone else comments.

        Show
        liuml07 Mingliang Liu added a comment - I think the patch makes sense; we need to make sure the condition is met before we do dependent operations and/or assertions. +1 I'll hold on commit this in 2 days in case someone else comments.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 26s trunk passed
        +1 compile 0m 50s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 59s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 47s trunk passed
        +1 javadoc 0m 41s trunk passed
        +1 mvninstall 0m 48s the patch passed
        +1 compile 0m 44s the patch passed
        +1 javac 0m 44s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 1s the patch passed
        +1 javadoc 0m 42s the patch passed
        -1 unit 79m 19s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        99m 25s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11105
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838706/HDFS-11105.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c406aed385cb 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 96f4392
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17547/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17547/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17547/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 26s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 41s trunk passed +1 mvninstall 0m 48s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 1s the patch passed +1 javadoc 0m 42s the patch passed -1 unit 79m 19s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 99m 25s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11105 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838706/HDFS-11105.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c406aed385cb 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 96f4392 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17547/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17547/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17547/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        linyiqun Yiqun Lin added a comment -

        I think this JIRA should be a part of the JIRA HDFS-10666. It also should use GenericTestUtils.waitFor to wait conditions. Ping Mingliang Liu, would you have a look? Thanks!

        Show
        linyiqun Yiqun Lin added a comment - I think this JIRA should be a part of the JIRA HDFS-10666 . It also should use GenericTestUtils.waitFor to wait conditions. Ping Mingliang Liu , would you have a look? Thanks!
        Hide
        linyiqun Yiqun Lin added a comment -

        Post a new patch to make a improvement based on v001 patch.

        Show
        linyiqun Yiqun Lin added a comment - Post a new patch to make a improvement based on v001 patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 3s trunk passed
        +1 compile 0m 54s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 1m 3s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 56s trunk passed
        +1 javadoc 0m 43s trunk passed
        +1 mvninstall 0m 56s the patch passed
        +1 compile 0m 53s the patch passed
        +1 javac 0m 53s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 55s the patch passed
        +1 javadoc 0m 44s the patch passed
        -1 unit 59m 8s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        80m 27s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-11105
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837124/HDFS-11105.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 302ab7ed1b98 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 19b3779
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17429/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17429/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17429/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 3s trunk passed +1 compile 0m 54s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 0m 43s trunk passed +1 mvninstall 0m 56s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 55s the patch passed +1 javadoc 0m 44s the patch passed -1 unit 59m 8s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 80m 27s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11105 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12837124/HDFS-11105.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 302ab7ed1b98 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 19b3779 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17429/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17429/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17429/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        linyiqun Yiqun Lin added a comment -

        Attach a initial patch, use GenericTestUtils.waitFor to make a improvement.

        Show
        linyiqun Yiqun Lin added a comment - Attach a initial patch, use GenericTestUtils.waitFor to make a improvement.

          People

          • Assignee:
            linyiqun Yiqun Lin
            Reporter:
            linyiqun Yiqun Lin
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development