Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11882

Precisely calculate acked length of striped block groups in updatePipeline

    XMLWordPrintableJSON

Details

    Description

      Some tests of erasure coding fails by the following exception. The following test was removed by HDFS-11823, however, this type of error can happen in real cluster.

      Running org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
      Tests run: 14, Failures: 0, Errors: 1, Skipped: 10, Time elapsed: 89.086 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
      testMultipleDatanodeFailure56(org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure)  Time elapsed: 38.831 sec  <<< ERROR!
      java.lang.IllegalStateException: null
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
      	at org.apache.hadoop.hdfs.DFSStripedOutputStream.updatePipeline(DFSStripedOutputStream.java:780)
      	at org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamerFailures(DFSStripedOutputStream.java:664)
      	at org.apache.hadoop.hdfs.DFSStripedOutputStream.closeImpl(DFSStripedOutputStream.java:1034)
      	at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:842)
      	at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
      	at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
      	at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:472)
      	at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTestWithMultipleFailure(TestDFSStripedOutputStreamWithFailure.java:381)
      	at org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.testMultipleDatanodeFailure56(TestDFSStripedOutputStreamWithFailure.java:245)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
      

      Attachments

        1. HDFS-11882.06.patch
          13 kB
          Andrew Wang
        2. HDFS-11882.05.patch
          12 kB
          Andrew Wang
        3. HDFS-11882.04.patch
          10 kB
          Andrew Wang
        4. HDFS-11882.03.patch
          10 kB
          Andrew Wang
        5. HDFS-11882.02.patch
          4 kB
          Akira Ajisaka
        6. HDFS-11882.regressiontest.patch
          2 kB
          Akira Ajisaka
        7. HDFS-11882.01.patch
          2 kB
          Akira Ajisaka

        Issue Links

          Activity

            People

              andrew.wang Andrew Wang
              aajisaka Akira Ajisaka
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: