Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-15678

camel-aws2-s3 multipart upload multiplies file by number of parts

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.5.0
    • Fix Version/s: 3.6.0, 3.4.5
    • Component/s: camel-aws2
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      Camel AWS2 S3 incorectly defines upload part content on multipart upload:
      org.apache.camel.component.aws2.s3.AWS2S3Producer:193

      for (int part = 1; filePosition < contentLength; part++) {
          ... 
          ...
          String etag = getEndpoint().getS3Client().uploadPart(uploadRequest, RequestBody.fromFile(filePayload)).eTag();
       ...
      
      }

       

      filePayload - is whole file to be uploaded. 

      In case when file size is bigger than partSize (which by default is 25MB) - uploaded amount of content to S3 will be file.size * number_of_parts

       

      Fix:

      • define RequsetBody from stream:
      InputStream inputStream = FileUtils.openInputStream(file);
      IOUtils.skip(inputStream, filePosition);
      
      completedParts.add(
              CompletedPart.builder()
                      .partNumber(part)
                      .eTag(amazonS3Client.uploadPart(uploadRequest, RequestBody.fromInputStream(inputStream, partSize)).eTag())
                      .build()
      ); 

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                acosentino Andrea Cosentino
                Reporter:
                cynicLT Kiril Nugmanov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: