[APEXMALHAR-2174] S3 File Reader reading more data than expected

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.5.0
• Assignee: Chaitanya Chebolu
• Reporter: Chaitanya Chebolu
• Created: 03/Aug/16 09:23
• Updated: 04/Sep/16 01:40
• Resolved: 09/Aug/16 06:08

Description

This is observed through the AWS billing.
Issue might be the S3InputStream.read() which is used in readEntity().

Reading the block can be achieved through the AmazonS3 api's. So, I am proposing the following solution:
```
GetObjectRequest rangeObjectRequest = new GetObjectRequest(
bucketName, key);
rangeObjectRequest.setRange(startByte, noOfBytes);
S3Object objectPortion = s3Client.getObject(rangeObjectRequest);
S3ObjectInputStream wrappedStream = objectPortion.getObjectContent();
byte[] record = ByteStreams.toByteArray(wrappedStream);

Advantages of this solution: Parallel read will work for all types of s3 file systems.

Issue Links

• links to: GitHub Pull Request #360