Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1366

OutOfMemory when InputStream referencing to big file is used as payload

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.0.0, 2.0.3
    • 2.2.0, 2.1.2
    • jclouds-blobstore
    • None
    • Linux and Windows

    Description

      If I use InputStream which source is large file (lets say 3GB) I am getting OOE. This is with default java VM options.
      Here is the code I am using to construct the blob:

       File bigFile = new File(file);
       try (InputStream inputStream = new FileInputStream(f)) {
                      Blob b = blobStore.blobBuilder(blobName)
                              .payload(inputStream).contentLength(f.length())
                              .contentDisposition(blobName)
                              .contentType(
                                      MediaType.OCTET_STREAM)
                              .userMetadata(ImmutableMap.of("a", "b", "test", "beta"))
                              .build();
                      blobStore.putBlob("test", bbbbb, multipart());
      }
      

      Stacktrace:

      java.lang.OutOfMemoryError: Java heap space
      	at org.jclouds.io.internal.BasePayloadSlicer$InputStreamPayloadIterator.getNextPayload(BasePayloadSlicer.java:101)
      	at org.jclouds.io.internal.BasePayloadSlicer$InputStreamPayloadIterator.next(BasePayloadSlicer.java:90)
      	at org.jclouds.io.internal.BasePayloadSlicer$InputStreamPayloadIterator.next(BasePayloadSlicer.java:63)
      	at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:363)
      	at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:349)
      	at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:262)
      

      If 'bigFile' is used as payload the bug is not reproducible.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            gaul Andrew Gaul
            dgyurdzhekliev Deyan
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment