Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11882

Precisely calculate acked length of striped block groups in updatePipeline

    XMLWordPrintableJSON

    Details

      Description

      Some tests of erasure coding fails by the following exception. The following test was removed by HDFS-11823, however, this type of error can happen in real cluster.

      Running org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
      Tests run: 14, Failures: 0, Errors: 1, Skipped: 10, Time elapsed: 89.086 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
      testMultipleDatanodeFailure56(org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure)  Time elapsed: 38.831 sec  <<< ERROR!
      java.lang.IllegalStateException: null
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
      	at org.apache.hadoop.hdfs.DFSStripedOutputStream.updatePipeline(DFSStripedOutputStream.java:780)
      	at org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamerFailures(DFSStripedOutputStream.java:664)
      	at org.apache.hadoop.hdfs.DFSStripedOutputStream.closeImpl(DFSStripedOutputStream.java:1034)
      	at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:842)
      	at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
      	at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
      	at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:472)
      	at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTestWithMultipleFailure(TestDFSStripedOutputStreamWithFailure.java:381)
      	at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.testMultipleDatanodeFailure56(TestDFSStripedOutputStreamWithFailure.java:245)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
      

        Attachments

        1. HDFS-11882.06.patch
          13 kB
          Andrew Wang
        2. HDFS-11882.05.patch
          12 kB
          Andrew Wang
        3. HDFS-11882.04.patch
          10 kB
          Andrew Wang
        4. HDFS-11882.03.patch
          10 kB
          Andrew Wang
        5. HDFS-11882.02.patch
          4 kB
          Akira Ajisaka
        6. HDFS-11882.regressiontest.patch
          2 kB
          Akira Ajisaka
        7. HDFS-11882.01.patch
          2 kB
          Akira Ajisaka

          Issue Links

            Activity

              People

              • Assignee:
                andrew.wang Andrew Wang
                Reporter:
                aajisaka Akira Ajisaka
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: