[HADOOP-14774] S3A case "testRandomReadOverBuffer" failed due to improper range parameter

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.9.0, 3.0.0-beta1
• Component/s: fs/s3

Environment

Hadoop 2.8.0
s3-compatible storage

• Assignee: Yonger
• Reporter: Yonger
• Created: 15/Aug/17 09:19
• Updated: 07/Sep/17 21:34
• Resolved: 07/Sep/17 18:59

Description

Running org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.821 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
testRandomReadOverBuffer(org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance)  Time elapsed: 2.605 sec  <<< FAILURE!
java.lang.AssertionError: expected:<8192> but was:<8193>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:555)
	at org.junit.Assert.assertEquals(Assert.java:542)
	at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.testRandomReadOverBuffer(ITestS3AInputStreamPerformance.java:533)

From log, the length of content is exceed than our expect:

2017-08-15 16:53:16,464 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(280)) - >> GET /test-aws-s3a/test/testReadOverBuffer.bin HTTP/1.1
2017-08-15 16:53:16,464 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Host: 10.0.2.254
2017-08-15 16:53:16,465 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2017-08-15 16:53:16,465 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Authorization: AWS4-HMAC-SHA256 Credential=JFDAM9KF9IY8S5P0JIV6/20170815/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;range;user-agent;x-amz-content-sha256;x-amz-date, Signature=42bce4a43d2b1bf6e6d599613c60812e6716514da4ef5b3839ef0566c31279ee
2017-08-15 16:53:16,466 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> X-Amz-Date: 20170815T085316Z
2017-08-15 16:53:16,466 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> User-Agent: Hadoop 2.8.0, aws-sdk-java/1.10.6 Linux/3.10.0-514.21.2.el7.x86_64 Java_HotSpot(TM)_64-Bit_Server_VM/25.131-b11/1.8.0_131
2017-08-15 16:53:16,467 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Range: bytes=0-8192
2017-08-15 16:53:16,467 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Content-Type: application/x-www-form-urlencoded; charset=utf-8
2017-08-15 16:53:16,467 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Connection: Keep-Alive
2017-08-15 16:53:16,473 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "HTTP/1.1 206 Partial Content[\r][\n]"
2017-08-15 16:53:16,475 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Content-Range: bytes 0-8192/32768[\r][\n]"
2017-08-15 16:53:16,476 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Content-Length: 8193[\r][\n]"
2017-08-15 16:53:16,477 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Accept-Ranges: bytes[\r][\n]"
2017-08-15 16:53:16,478 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Last-Modified: Tue, 15 Aug 2017 08:51:39 GMT[\r][\n]"
2017-08-15 16:53:16,479 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "ETag: "e7191764798ba504d6671d4c434d2f4d"[\r][\n]"
2017-08-15 16:53:16,480 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "x-amz-request-id: tx00000000000000000001e-005992b67e-27a45-default[\r][\n]"
2017-08-15 16:53:16,481 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Content-Type: application/octet-stream[\r][\n]"
2017-08-15 16:53:16,482 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Date: Tue, 15 Aug 2017 08:53:18 GMT[\r][\n]"
2017-08-15 16:53:16,483 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "[\r][\n]"