Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1465

Error with multipart upload to S3 for large files

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.1
    • Fix Version/s: None
    • Component/s: jclouds-blobstore
    • Labels:
      None

      Description

      Hi,

      We have a problem with multipart upload to S3 for files above or even to 20GB. The exception is here:

      2018-10-31 18:04:32,804 ERROR [EventQueueWorker-ThreadPool-16] s3.jclouds.ssl.CustomJavaUrlHttpCommandExecutorService - error after writing 24576/16800342 bytes to "some s3 container" java.io.IOException: Error writing request body to server at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:3536) at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3519) at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53) at com.google.common.io.ByteStreams.copy(ByteStreams.java:179) at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService. java:298)

       

      We are using the last jclouds version (2.1.1). As we make transfers to S3 on a daily basis, the expected behavior is: files to be successfully uploaded. Now they fail while uploading the first part of the multipart file. I have examined similar issue: http://mail-archives.apache.org/mod_mbox/jclouds-dev/201504.mbox/%3CCAHtp1K2vRaZxQCBAR8ZCjXoXxbnz58hNZCtFmmrNjYKSJ7dHgw@mail.gmail.com%3E and it seems that it is not resolved. Have you identify it and start working on it? If not can you suggest any workaround?

      Best Regards,

      Blago

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              blagolaj Blagoi Anastasov
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: