Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1549

Multipart upload fails on AWS S3 buckets with Object Lock and retention

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.2.0
    • None
    • jclouds-blobstore
    • None

    Description

      AWS S3 buckets with object lock and a default object retention period (governance/compliance) require the content MD5 header for PUT operations (https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html)
      When multipart uploading a blob into such a bucket, a response is returned indicating that the content MD5 header is missing.

      The following snippet:

      BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
          .endpoint("https://s3.amazonaws.com")
          .credentials("...", "...")
          .buildView(BlobStoreContext.class);
      BlobStore blobStore = blobStoreContext.getBlobStore();
      ByteSource payloadByteSource = ByteSource.wrap("payloaddata".getBytes(StandardCharsets.UTF_8));
      Blob blob = blobStore.blobBuilder("multipart-uploaded-blob")
          .userMetadata(Collections.emptyMap())
          .payload(payloadByteSource)
          .contentLength(payloadByteSource.size())
          .contentMD5(payloadByteSource.hash(Hashing.md5()))
          .build();
      blobStore.putBlob("my-temp-object-locked-bucket", blob, PutOptions.Builder.multipart());
      blobStoreContext.close(); 
      

       

      Results in the exception:

      com.google.common.util.concurrent.UncheckedExecutionException: org.jclouds.aws.AWSResponseException: request PUT https://my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com/multipart-uploaded-blob?partNumber=1&uploadId=7SXM.6rCAh_zyPYoF1KPyc86niKrkvUqoke00mEbJrAj9pdmp8Cf0cK6uGbxVBjEJNXkO8JVC6Yb5aQKI30WP4eohjdfFQOtY888JlUm6eOFKWy8lsmNZ5fBgRcPCdMD HTTP/1.1 failed with code 400, error: AWSError{requestId='91FBACF8DF0AE5EB', requestToken='1ce75Bg73qNTIjUfGiI4QjEIarPEEKm37NMNvt7dLAM68DHYsqPahR1JKVOobkPHX/fSidHbdmM=', code='InvalidRequest', message='Content-MD5 HTTP header is required for Put Part requests with Object Lock parameters', context='{HostId=1ce75Bg73qNTIjUfGiI4QjEIarPEEKm37NMNvt7dLAM68DHYsqPahR1JKVOobkPHX/fSidHbdmM=}'}	at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1525)
      	at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1511)
      	at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:395)
      	at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:349)
      	at org.jclouds.aws.s3.blobstore.AWSS3BlobStore.putBlob(AWSS3BlobStore.java:79)
      	at io.model9.backup.common.objectstorage.MultipartMd5Test.multipartMd5(MultipartMd5Test.java:34)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
      	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
      	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
      	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
      Caused by: org.jclouds.aws.AWSResponseException: request PUT https://my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com/multipart-uploaded-blob?partNumber=1&uploadId=7SXM.6rCAh_zyPYoF1KPyc86niKrkvUqoke00mEbJrAj9pdmp8Cf0cK6uGbxVBjEJNXkO8JVC6Yb5aQKI30WP4eohjdfFQOtY888JlUm6eOFKWy8lsmNZ5fBgRcPCdMD HTTP/1.1 failed with code 400, error: AWSError{requestId='91FBACF8DF0AE5EB', requestToken='1ce75Bg73qNTIjUfGiI4QjEIarPEEKm37NMNvt7dLAM68DHYsqPahR1JKVOobkPHX/fSidHbdmM=', code='InvalidRequest', message='Content-MD5 HTTP header is required for Put Part requests with Object Lock parameters', context='{HostId=1ce75Bg73qNTIjUfGiI4QjEIarPEEKm37NMNvt7dLAM68DHYsqPahR1JKVOobkPHX/fSidHbdmM=}'}
      	at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:76)
      	at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
      	at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:138)
      	at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:107)
      	at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
      	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
      	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
      	at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
      	at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
      	at com.sun.proxy.$Proxy44.uploadPart(Unknown Source)
      	at org.jclouds.s3.blobstore.S3BlobStore.uploadMultipartPart(S3BlobStore.java:391)
      	at org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:415)
      	at org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:402)
      	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
      	at java.util.concurrent.FutureTask.run(FutureTask.java)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      	at java.lang.Thread.getStackTrace(Thread.java:1559)
      	at org.jclouds.blobstore.internal.BaseBlobStore.putMultipartBlob(BaseBlobStore.java:393)
      	... 25 more
      

       

      Here are the requests (URL and headers) initiated by JClouds in the above snippet:

      GET >> https://s3.amazonaws.com/my-temp-object-locked-bucket?location
      {Host=[s3.amazonaws.com], x-amz-content-sha256=[e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855], X-Amz-Date=[20200721T160746Z], Authorization=[AWS4-HMAC-SHA256 Credential=..., SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=db96f64af0bc4457ceda679805b6bb70656494b0ae5ae4d6501f1b19847dc31a]}
      POST >> https://my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com/multipart-uploaded-blob?uploads
      {Content-Type=[application/unknown], Content-MD5=[nirLkofY1bN5hCACAirgRA==], Host=[my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com], x-amz-content-sha256=[e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855], X-Amz-Date=[20200721T160746Z], Authorization=[AWS4-HMAC-SHA256 Credential=..., SignedHeaders=content-md5;content-type;host;x-amz-content-sha256;x-amz-date, Signature=fdb6a51c9cfbe116f8da033e079d4a9033a63b1d7a3abe8256546b2363f2dddb]}
      PUT >> https://my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com/multipart-uploaded-blob?partNumber=1&uploadId=7SXM.6rCAh_zyPYoF1KPyc86niKrkvUqoke00mEbJrAj9pdmp8Cf0cK6uGbxVBjEJNXkO8JVC6Yb5aQKI30WP4eohjdfFQOtY888JlUm6eOFKWy8lsmNZ5fBgRcPCdMD
      {Content-Type=[application/unknown], Content-Length=[11], Host=[my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com], x-amz-content-sha256=[9e8f1b13c357dd00b99dd1760be67a3fdeb3ed1e3605307e549ab74c483a43fa], X-Amz-Date=[20200721T160746Z], Authorization=[AWS4-HMAC-SHA256 Credential=.., SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=9aec5a51bbffaa95321a054d8d6aad40b7c7d87d6967562592f8aaf51bb5e7aa]}
      DELETE >> https://my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com/multipart-uploaded-blob?uploadId=7SXM.6rCAh_zyPYoF1KPyc86niKrkvUqoke00mEbJrAj9pdmp8Cf0cK6uGbxVBjEJNXkO8JVC6Yb5aQKI30WP4eohjdfFQOtY888JlUm6eOFKWy8lsmNZ5fBgRcPCdMD
      {Host=[my-temp-object-locked-bucket.s3-eu-central-1.amazonaws.com], x-amz-content-sha256=[e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855], X-Amz-Date=[20200721T160746Z], Authorization=[AWS4-HMAC-SHA256 Credential=..., SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=f2e53347d054d7c4a08eb1d62b1ff828d90135d0563d85ee8547626788dfd290]}
      

      It looks like the blob's MD5 hash is added to the POST to initiate the multipart upload but is missing from the part's PUT itself.

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            roded roded
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: