[CAMEL-15678] camel-aws2-s3 multipart upload multiplies file by number of parts

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.6.0, 3.4.5
• Component/s: camel-aws2
• Estimated Complexity: Unknown
• Assignee: Andrea Cosentino
• Reporter: Kiril Nugmanov
• Created: 13/Oct/20 10:45
• Updated: 13/Oct/20 15:59
• Resolved: 13/Oct/20 15:59

Description

Camel AWS2 S3 incorectly defines upload part content on multipart upload:
org.apache.camel.component.aws2.s3.AWS2S3Producer:193

for (int part = 1; filePosition < contentLength; part++) {
    ... 
    ...
    String etag = getEndpoint().getS3Client().uploadPart(uploadRequest, RequestBody.fromFile(filePayload)).eTag();
 ...

}

 

filePayload - is whole file to be uploaded. 

In case when file size is bigger than partSize (which by default is 25MB) - uploaded amount of content to S3 will be file.size * number_of_parts

 

Fix:

• define RequsetBody from stream:

InputStream inputStream = FileUtils.openInputStream(file);
IOUtils.skip(inputStream, filePosition);

completedParts.add(
        CompletedPart.builder()
                .partNumber(part)
                .eTag(amazonS3Client.uploadPart(uploadRequest, RequestBody.fromInputStream(inputStream, partSize)).eTag())
                .build()
); 

Issue Links

• links to: GitHub Pull Request #4433