Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11730

Regression: s3n read failure recovery broken

    Details

    • Target Version/s:

      Description

      s3n attempts to read again when it encounters IOException during read. But the current logic does not reopen the connection, thus, it ends up with no-op, and committing the wrong(truncated) output.

      Here's a stack trace as an example.

      2015-03-13 20:17:24,835 [TezChild] INFO org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor - Starting output org.apache.tez.mapreduce.output.MROutput@52008dbd to vertex scope-12
      2015-03-13 20:17:24,866 [TezChild] DEBUG org.jets3t.service.impl.rest.httpclient.HttpMethodReleaseInputStream - Released HttpMethod as its response data stream threw an exception
      org.apache.http.ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 296587138; received: 155648
      at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:184)
      at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:138)
      at org.jets3t.service.io.InterruptableInputStream.read(InterruptableInputStream.java:78)
      at org.jets3t.service.impl.rest.httpclient.HttpMethodReleaseInputStream.read(HttpMethodReleaseInputStream.java:146)
      at org.apache.hadoop.fs.s3native.NativeS3FileSystem$NativeS3FsInputStream.read(NativeS3FileSystem.java:145)
      at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
      at java.io.DataInputStream.read(DataInputStream.java:100)
      at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
      at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
      at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
      at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:185)
      at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:259)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:204)
      at org.apache.tez.mapreduce.lib.MRReaderMapReduce.next(MRReaderMapReduce.java:116)
      at org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POSimpleTezLoad.getNextTuple(POSimpleTezLoad.java:106)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:307)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:246)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:307)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNextTuple(POFilter.java:91)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:307)
      at org.apache.pig.backend.hadoop.executionengine.tez.plan.operator.POStoreTez.getNextTuple(POStoreTez.java:117)
      at org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:313)
      at org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:192)
      at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
      at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
      at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
      at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      2015-03-13 20:17:24,867 [TezChild] INFO org.apache.hadoop.fs.s3native.NativeS3FileSystem - Received IOException while reading 'user/hadoop/tsato/readlarge/input/cloudian-s3.log.20141119', attempting to reopen.
      2015-03-13 20:17:24,867 [TezChild] DEBUG org.jets3t.service.impl.rest.httpclient.HttpMethodReleaseInputStream - Released HttpMethod as its response data stream is fully consumed
      2015-03-13 20:17:24,868 [TezChild] INFO org.apache.tez.dag.app.TaskAttemptListenerImpTezDag - Commit go/no-go request from attempt_1426245338920_0001_1_00_000004_0
      2015-03-13 20:17:24,868 [TezChild] INFO org.apache.tez.dag.app.dag.impl.TaskImpl - attempt_1426245338920_0001_1_00_000004_0 given a go for committing the task output.

      It seems this is a regression, which was introduced by the following optimizations.

      https://issues.apache.org/jira/browse/HADOOP-10589
      https://issues.apache.org/jira/browse/HADOOP-10457

      Also, test cases should be reviewed so that it covers this scenario.

      1. HADOOP-11730-002.patch
        6 kB
        Steve Loughran
      2. HADOOP-11730-branch-2.6.0.001.patch
        5 kB
        Takenori Sato

        Issue Links

          Activity

          Hide
          tsato Takenori Sato added a comment -

          The first proposal without the test case.

          2015-03-20 12:05:08,473 [TezChild] INFO org.apache.hadoop.fs.s3native.NativeS3FileSystem - Received IOException while reading 'user/hadoop/tsato/readlarge/input/cloudian-s3.log.20141119', attempting to reopen.
          2015-03-20 12:05:08,473 [TezChild] DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Retrieving All information for bucket shared and object user/hadoop/tsato/readlarge/input/cloudian-s3.log.20141119

          Verified manually that it reopens a new connection after IOException.

          Show
          tsato Takenori Sato added a comment - The first proposal without the test case. 2015-03-20 12:05:08,473 [TezChild] INFO org.apache.hadoop.fs.s3native.NativeS3FileSystem - Received IOException while reading 'user/hadoop/tsato/readlarge/input/cloudian-s3.log.20141119', attempting to reopen. 2015-03-20 12:05:08,473 [TezChild] DEBUG org.jets3t.service.impl.rest.httpclient.RestStorageService - Retrieving All information for bucket shared and object user/hadoop/tsato/readlarge/input/cloudian-s3.log.20141119 Verified manually that it reopens a new connection after IOException.
          Hide
          tsato Takenori Sato added a comment -

          The first patch with the updated test case.

          Show
          tsato Takenori Sato added a comment - The first patch with the updated test case.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 32s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace.
          +1 javac 7m 27s There were no new javac warning messages.
          +1 javadoc 9m 36s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 5m 28s The applied patch generated 1 additional checkstyle issues.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 0m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 tools/hadoop tests 0m 15s Tests passed in hadoop-aws.
              40m 33s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12705845/HADOOP-11730-branch-2.6.0.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 189a63a
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/artifact/patchprocess/whitespace.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-aws test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/artifact/patchprocess/testrun_hadoop-aws.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/testReport/
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. +1 javac 7m 27s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 5m 28s The applied patch generated 1 additional checkstyle issues. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 tools/hadoop tests 0m 15s Tests passed in hadoop-aws.     40m 33s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12705845/HADOOP-11730-branch-2.6.0.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 189a63a whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/artifact/patchprocess/whitespace.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/artifact/patchprocess/checkstyle-result-diff.txt hadoop-aws test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/artifact/patchprocess/testrun_hadoop-aws.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/testReport/ Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6167/console This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1 committing

          Here's the patch in sync with trunk; it also incorporates HADOOP-11851 in the close logic, as they go hand in hand. We can't have the recovery process damaged by ConnectionReset exceptions being picked up while it closes the old stream.

          Show
          stevel@apache.org Steve Loughran added a comment - +1 committing Here's the patch in sync with trunk; it also incorporates HADOOP-11851 in the close logic, as they go hand in hand. We can't have the recovery process damaged by ConnectionReset exceptions being picked up while it closes the old stream.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          patch applied; tested against s3 EU.

          Given the nature of these problems, it may be good to start thinking about whether we can do things with better simulate failures; the test here is a good start, though we may want more complex policies...mockito might be the tool to reach for.

          Show
          stevel@apache.org Steve Loughran added a comment - patch applied; tested against s3 EU. Given the nature of these problems, it may be good to start thinking about whether we can do things with better simulate failures; the test here is a good start, though we may want more complex policies...mockito might be the tool to reach for.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7653 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7653/)
          HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7653 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7653/ ) HADOOP-11730 . Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 33s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 27s There were no new javac warning messages.
          +1 javadoc 9m 33s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 7m 43s The applied patch generated 1 additional checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 0m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 tools/hadoop tests 0m 14s Tests passed in hadoop-aws.
              42m 39s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12727699/HADOOP-11730-002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 416b843
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-aws test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/artifact/patchprocess/testrun_hadoop-aws.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/testReport/
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 27s There were no new javac warning messages. +1 javadoc 9m 33s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 7m 43s The applied patch generated 1 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 38s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 tools/hadoop tests 0m 14s Tests passed in hadoop-aws.     42m 39s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727699/HADOOP-11730-002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 416b843 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/artifact/patchprocess/checkstyle-result-diff.txt hadoop-aws test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/artifact/patchprocess/testrun_hadoop-aws.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/testReport/ Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6170/console This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/)
          HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/ ) HADOOP-11730 . Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/)
          HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591)

          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/ ) HADOOP-11730 . Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/)
          HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/ ) HADOOP-11730 . Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/)
          HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591)

          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/ ) HADOOP-11730 . Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/)
          HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591)

          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/ ) HADOOP-11730 . Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/)
          HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591)

          • hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/ ) HADOOP-11730 . Regression: s3n read failure recovery broken. (Takenori Sato via stevel) (stevel: rev 19262d99ebbbd143a7ac9740d3a8e7c842b37591) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3native/NativeS3FileSystemContractBaseTest.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Pulled this into 2.6.1 after Akira Ajisaka verified that the patch applies cleanly. Ran compilation and TestInMemoryNativeS3FileSystemContract before the push.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.6.1 after Akira Ajisaka verified that the patch applies cleanly. Ran compilation and TestInMemoryNativeS3FileSystemContract before the push.

            People

            • Assignee:
              tsato Takenori Sato
              Reporter:
              tsato Takenori Sato
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development