Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-626

OutOfMemoryError with FileSystem provider, large content from non-repeatable input streams

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.3
    • Fix Version/s: 1.8.0
    • Component/s: jclouds-blobstore
    • Labels:
      None
    • Environment:
      Mac OS X 10.9.4, Oracle Java 1.7.0_55; RHEL 6.5, IBM Java

      Description

      The following (incomplete) piece of code dies on my systems with a stacktrace (given below):

       public void sendLargeFilesOnlyJclouds() throws Exception
          {
              LOG.info("Working with {}", p);
              Environment env = Environment.builder().setName("SomeName").addProperties(p).build();
      
              String providerName = env.getConnectionModules().get(0).getName();
              String provider = checkNotNull(env.getProperty(providerName + ".provider"), new ICStoreException("No provider specified"));
      
              Iterable<? extends Module> modules =
                      ImmutableSet.of(
                              new SLF4JLoggingModule()
                      );
      
              if(provider.equals("s3"))
              {
                  provider = "aws-s3";
                  modules =
                          ImmutableSet.<Module>builder()
                                  .addAll(modules)
                                  .add(new ICStoreMultipartUploadModule())
                                  .build();
              }
              Properties properties = new Properties();
              // store provider credentials in properties
              properties.setProperty(provider + ".identity", env.getProperty( providerName + ".identity", ""));
              properties.setProperty(provider + ".credential", env.getProperty(providerName + ".credential", ""));
              properties.putAll(env.getPropertiesForModule(providerName));
      
              final BlobStoreContext blobStoreContext = ContextBuilder.newBuilder(provider)
                      .overrides(properties)
                      .modules(modules)
                      .buildView(BlobStoreContext.class);
      
              final BlobStore bs = blobStoreContext.getBlobStore();
      
              bs.createContainerInLocation(null, "mycontainer");
      
              for (int i = 0; i < num_invocations; i++)
              {
                  InputStream fis = new FakeInputStream(FAKE_IS_SEED, blobSize);
                  Blob blob = bs.blobBuilder("myblob").payload(fis).build();
      
                  LOG.info("Invocation number {}", i);
                  long startTime = System.currentTimeMillis();
                  bs.putBlob("mycontainer", blob);
                  long stopTime = System.currentTimeMillis();
      
                  LOG.info("Running time for one blob is {}", stopTime - startTime);
              }
          }
      

      Where FakeInputStream is an input stream that produces a deterministic output (though method read(), of a given size, and Environment is just a smarter container for Properties.

      The resulting stack trace follows, and I get it during the first iteration:

      2014-07-14@12:03:48.653 I            [main  ][        ] soTest:sendLargeFilesOnlyJclouds - Invocation number 0
      java.lang.OutOfMemoryError: Requested array size exceeds VM limit
      Dumping heap to java_pid8693.hprof ...
      Heap dump file created [2145617933 bytes in 85.677 secs]
      Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
      	at java.util.Arrays.copyOf(Arrays.java:2271)
      	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
      	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
      	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
      	at com.google.common.io.ByteStreams.copy(ByteStreams.java:173)
      	at com.google.common.io.ByteStreams.toByteArray(ByteStreams.java:214)
      	at com.google.common.io.ByteSource.read(ByteSource.java:266)
      	at com.google.common.io.ByteStreams.toByteArray(ByteStreams.java:226)
      	at org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.putBlob(FilesystemStorageStrategyImpl.java:205)
      	at org.jclouds.blobstore.LocalAsyncBlobStore.putBlob(LocalAsyncBlobStore.java:397)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37)
      	at com.sun.proxy.$Proxy45.putBlob(Unknown Source)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.google.common.reflect.Invokable$MethodInvokable.invokeInternal(Invokable.java:197)
      	at com.google.common.reflect.Invokable.invoke(Invokable.java:102)
      	at org.jclouds.rest.internal.InvokeAndCallGetOnFutures.apply(InvokeAndCallGetOnFutures.java:68)
      	at org.jclouds.rest.internal.InvokeAndCallGetOnFutures.apply(InvokeAndCallGetOnFutures.java:40)
      	at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
      	at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
      	at com.sun.proxy.$Proxy46.putBlob(Unknown Source)
      	at com.ibm.icstore.stress.LargeFilesTest.sendLargeFilesOnlyJclouds(LargeFilesTest.java:153)
      	at com.ibm.icstore.stress.LargeFilesTest.main(LargeFilesTest.java:163)
      

      I suspect the problem is with the following piece of code in the method putBlob in FilesystemStorageStrategyImpl.java:

               else {
                  if (!payload.isRepeatable()) {
                     payload = Payloads.newPayload(ByteStreams.toByteArray(payload));
                  }
                  Files.copy(payload, outputFile);
               }
      

      Judging by the previous fix for the issue (didn't fix it), the culprit seems to be:
      Payloads.newPayload(ByteStreams.toByteArray(payload));

      Why not completely ditch that line, and just go for Files.copy(), as it accepts an InputStream, and doesn't require a repeatable IS?

        Attachments

          Activity

            People

            • Assignee:
              gaul Andrew Gaul
              Reporter:
              knl Nikola Knezevic
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: