Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9383

TestByteArrayManager#testByteArrayManager fails

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This was seen in the trunk builds
      https://builds.apache.org/job/Hadoop-Hdfs-trunk

      Running org.apache.hadoop.hdfs.util.TestByteArrayManager
      Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 6.539 sec <<< FAILURE!
       - in org.apache.hadoop.hdfs.util.TestByteArrayManager
      testByteArrayManager(org.apache.hadoop.hdfs.util.TestByteArrayManager)  Time elapsed: 5.409 sec  <<< FAILURE!
      java.lang.AssertionError: expected null, but was:<[32: 2/64, free=5]>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotNull(Assert.java:664)
      	at org.junit.Assert.assertNull(Assert.java:646)
      	at org.junit.Assert.assertNull(Assert.java:656)
      	at org.apache.hadoop.hdfs.util.TestByteArrayManager.testByteArrayManager(TestByteArrayManager.java:384)
      
      1. h9383_20151107.patch
        0.9 kB
        Tsz Wo Nicholas Sze
      2. hdfs-9383.log
        98 kB
        Kihwal Lee

        Issue Links

          Activity

          Hide
          kihwal Kihwal Lee added a comment -

          Test log attached.

          Show
          kihwal Kihwal Lee added a comment - Test log attached.
          Hide
          kihwal Kihwal Lee added a comment -

          Tsz Wo Nicholas Sze, do you think it is just a test issue or actual bug in the code?

          Show
          kihwal Kihwal Lee added a comment - Tsz Wo Nicholas Sze , do you think it is just a test issue or actual bug in the code?
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          Thanks for filing the issue. Let me take a look.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - Thanks for filing the issue. Let me take a look.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          After the change in HDFS-8971, the debug log (i.e. the posted log) became very long and very hard to read. We probably need to fix the log first.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - After the change in HDFS-8971 , the debug log (i.e. the posted log) became very long and very hard to read. We probably need to fix the log first.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          I suspect that this is a test bug: the counter had been once above threshold but got reset later. So the count was within threshold during assertion. We may change countResetTimePeriodMs to a longer period in order to fix the test bug.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - I suspect that this is a test bug: the counter had been once above threshold but got reset later. So the count was within threshold during assertion. We may change countResetTimePeriodMs to a longer period in order to fix the test bug.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          HDFS-9398 fixed the debug message format.

          I am going to increase the countResetTimePeriodMs in the test.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - HDFS-9398 fixed the debug message format. I am going to increase the countResetTimePeriodMs in the test.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          h9383_20151107.patch: changes the reset time from 1s to 10s.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - h9383_20151107.patch: changes the reset time from 1s to 10s.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 18s trunk passed
          +1 compile 0m 27s trunk passed with JDK v1.8.0_66
          +1 compile 0m 26s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 11s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 57s trunk passed
          +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 24s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 30s the patch passed
          +1 compile 0m 28s the patch passed with JDK v1.8.0_66
          +1 javac 0m 28s the patch passed
          +1 compile 0m 25s the patch passed with JDK v1.7.0_79
          +1 javac 0m 25s the patch passed
          +1 checkstyle 0m 10s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 9s the patch passed
          +1 javadoc 0m 23s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 25s the patch passed with JDK v1.7.0_79
          +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 33s Patch does not generate ASF License warnings.
          17m 12s



          Subsystem Report/Notes
          Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-07
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771181/h9383_20151107.patch
          JIRA Issue HDFS-9383
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 9ce3ebfe2f6b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 9614dea
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13430/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13430/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 18s trunk passed +1 compile 0m 27s trunk passed with JDK v1.8.0_66 +1 compile 0m 26s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 11s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 57s trunk passed +1 javadoc 0m 23s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 24s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 30s the patch passed +1 compile 0m 28s the patch passed with JDK v1.8.0_66 +1 javac 0m 28s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_79 +1 javac 0m 25s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 9s the patch passed +1 javadoc 0m 23s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 25s the patch passed with JDK v1.7.0_79 +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 33s Patch does not generate ASF License warnings. 17m 12s Subsystem Report/Notes Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-07 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771181/h9383_20151107.patch JIRA Issue HDFS-9383 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 9ce3ebfe2f6b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 9614dea Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13430/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13430/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          +1.

          Show
          kihwal Kihwal Lee added a comment - +1.
          Hide
          kihwal Kihwal Lee added a comment -

          Thanks for analyzing the bug and providing the patch, Nicholas. I've committed this to trunk, branch-2 and branch-2.7. git cherry-pick figured out where the file moved and did the right thing. I am impressed.

          Show
          kihwal Kihwal Lee added a comment - Thanks for analyzing the bug and providing the patch, Nicholas. I've committed this to trunk, branch-2 and branch-2.7. git cherry-pick figured out where the file moved and did the right thing. I am impressed.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8778 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8778/)
          HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8778 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8778/ ) HDFS-9383 . TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde) hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #645 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/645/)
          HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #645 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/645/ ) HDFS-9383 . TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde) hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Mapreduce-trunk #2586 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2586/)
          HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Mapreduce-trunk #2586 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2586/ ) HDFS-9383 . TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Hdfs-trunk #2526 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2526/)
          HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Hdfs-trunk #2526 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2526/ ) HDFS-9383 . TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #587 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/587/)
          HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #587 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/587/ ) HDFS-9383 . TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde) hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Yarn-trunk-Java8 #656 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/656/)
          HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Yarn-trunk-Java8 #656 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/656/ ) HDFS-9383 . TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde) hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Yarn-trunk #1379 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1379/)
          HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Yarn-trunk #1379 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1379/ ) HDFS-9383 . TestByteArrayManager#testByteArrayManager fails. Contributed (kihwal: rev ef926b2e3824475581454c1e17a0d7c94529efde) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/util/TestByteArrayManager.java
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              szetszwo Tsz Wo Nicholas Sze
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development