Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1401

S3 SignatureDoesNotMatch when listing keys using prefix with ampersand

    Details

    • Type: Bug
    • Status: Open
    • Priority: Trivial
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: jclouds-blobstore
    • Labels:

      Description

      When using a prefix with ampersand & the list request fails with this exception.
      The AWS SDK handles it fine

          public void testAmpersand() throws Exception {
              String key = getPrefix() + "xxx#?:&$'\"<>čॐ";
      
              try {
                  blobStore.putBlob(getContainer(), blobStore.blobBuilder(key).payload("test").build());
      
                  final AmazonS3 s3 = AmazonS3ClientBuilder.defaultClient();
                  // works fine
                  ListObjectsV2Result result = s3.listObjectsV2(getContainer(), key);
                  List<S3ObjectSummary> objects = result.getObjectSummaries();
                  assertThat(objects, not(empty()));
      
                  // fails with
                  // org.jclouds.rest.AuthorizationException: The request signature we calculated does not match the signature
                  // you provided. Check your key and signing method.
                  PageSet<? extends StorageMetadata> list = blobStore.list(getContainer(), prefix(key));
                  assertThat(list, not(empty()));
              } finally {
                  blobStore.removeBlob(getContainer(), key);
              }
          }
      
      org.jclouds.rest.AuthorizationException: The request signature we calculated does not match the signature you provided. Check your key and signing method.
      at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.refineException(ParseAWSErrorFromXmlContent.java:122)
      at org.jclouds.s3.handlers.ParseS3ErrorFromXmlContent.refineException(ParseS3ErrorFromXmlContent.java:97)
      at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:89)
      at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
      at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:140)
      at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:109)
      at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
      at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
      at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
      at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
      at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
      at com.sun.proxy.$Proxy56.listBucket(Unknown Source)
      at org.jclouds.s3.blobstore.S3BlobStore.list(S3BlobStore.java:175)
      at io.jenkins.plugins.artifact_manager_s3.JCloudsBlobStore.isDirectory(JCloudsBlobStore.java:160)
      at io.jenkins.plugins.artifact_manager_s3.JCloudsBlobStoreTest.isDirectory(JCloudsBlobStoreTest.java:132)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      at org.junit.rules.RunRules.evaluate(RunRules.java:20)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
      at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
      Caused by: org.jclouds.aws.AWSResponseException: request GET https://bucket_name.s3.amazonaws.com/?delimiter=/&prefix=jclouds-test/2018-04-05T12%3A11%3A54.228Z/xxx%23%3F%3A%26%24%27%22%3C%3E%C4%8D%E0%A5%90/ HTTP/1.1 failed with code 403, error: AWSError{requestId='331FD9D8C8686E14', requestToken='8Qk2cHXOIOT1KtGsehcbhhli5N4DyS/G0UyK06aG1FSo6cdby/6pOZdeY2TKgZ+gqcsr//to5Og=', code='SignatureDoesNotMatch', message='The request signature we calculated does not match the signature you provided. Check your key and signing method.', context='{CanonicalRequestBytes=47 45 54 0a 2f 0a 64 65 6c 69 6d 69 74 65 72 3d 25 32 46 26 70 72 65 66 69 78 3d 6a 63 6c 6f 75 64 73 2d 74 65 73 74 25 32 46 32 30 31 38 2d 30 34 2d 30 35 54 31 32 25 33 41 31 31 25 33 41 35 34 2e 32 32 38 5a 25 32 46 78 78 78 25 32 33 25 33 46 25 33 41 25 32 36 25 32 34 25 32 37 25 32 32 25 33 43 25 33 45 25 43 34 25 38 44 25 45 30 25 41 35 25 39 30 25 32 46 0a 68 6f 73 74 3a 63 6c 6f 75 64 62 65 65 73 2d 6b 75 62 65 72 6e 65 74 65 73 2d 61 72 63 68 2e 73 33 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35 0a 78 2d 61 6d 7a 2d 64 61 74 65 3a 32 30 31 38 30 34 30 35 54 31 32 31 31 35 36 5a 0a 78 2d 61 6d 7a 2d 73 65 63 75 72 69 74 79 2d 74 6f 6b 65 6e 3a 46 51 6f 44 59 58 64 7a 45 48 55 61 44 4d 5a 2f 61 44 51 47 31 45 4d 7a 74 46 78 4b 4d 43 4c 72 41 57 57 66 62 51 6f 53 63 46 6b 79 4b 4b 70 6b 45 71 34 2f 31 6f 2b 55 6d 42 77 66 72 48 79 48 71 55 34 58 7a 65 4d 2b 4e 67 38 79 63 59 76 69 69 4b 49 77 45 45 77 55 6b 58 74 39 30 74 6f 44 4a 34 70 6d 7a 37 39 34 48 33 30 58 44 57 59 74 62 62 49 2b 51 54 62 63 48 65 50 61 6e 37 45 43 4e 42 71 69 6a 53 61 54 51 58 68 6e 6b 43 38 65 6f 38 73 75 58 70 55 46 42 44 31 2b 4a 42 77 6a 58 6f 79 32 62 78 67 5a 6c 4b 6f 35 78 39 38 64 74 41 64 4d 45 34 36 5a 34 58 65 6a 70 6e 49 35 48 75 79 79 50 62 4d 58 57 55 79 42 69 71 36 50 43 44 68 6e 6b 42 59 6e 4f 64 59 71 37 7a 52 4b 47 62 30 56 2b 5a 76 39 5a 56 6b 4c 7a 47 34 47 4d 73 49 37 42 51 4c 66 4e 32 6f 46 58 70 42 2b 48 39 6e 36 6b 63 32 39 75 45 35 6b 76 48 72 30 34 4b 4e 36 70 64 4e 52 52 2f 4b 46 36 6d 71 4a 39 72 4f 57 34 37 4e 66 44 66 56 69 38 61 39 62 6c 4d 62 6c 77 50 59 47 77 48 4a 4f 36 66 64 59 44 38 33 69 53 61 58 54 79 39 78 44 62 47 34 53 68 36 48 64 62 52 53 59 2b 46 45 6f 6c 36 4f 59 31 67 55 3d 0a 0a 68 6f 73 74 3b 78 2d 61 6d 7a 2d 63 6f 6e 74 65 6e 74 2d 73 68 61 32 35 36 3b 78 2d 61 6d 7a 2d 64 61 74 65 3b 78 2d 61 6d 7a 2d 73 65 63 75 72 69 74 79 2d 74 6f 6b 65 6e 0a 65 33 62 30 63 34 34 32 39 38 66 63 31 63 31 34 39 61 66 62 66 34 63 38 39 39 36 66 62 39 32 34 32 37 61 65 34 31 65 34 36 34 39 62 39 33 34 63 61 34 39 35 39 39 31 62 37 38 35 32 62 38 35 35, CanonicalRequest=GET
      /
      delimiter=%2F&prefix=jclouds-test%2F2018-04-05T12%3A11%3A54.228Z%2Fxxx%23%3F%3A%26%24%27%22%3C%3E%C4%8D%E0%A5%90%2F
      host:cloudbees-kubernetes-arch.s3.amazonaws.com
      x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
      x-amz-date:20180405T121156Z
      x-amz-security-token:FQoDYXdzEHUaDMZ/aDQG1EMztFxKMCLrAWWfbQoScFkyKKpkEq4/1o+UmBwfrHyHqU4XzeM+Ng8ycYviiKIwEEwUkXt90toDJ4pmz794H30XDWYtbbI+QTbcHePan7ECNBqijSaTQXhnkC8eo8suXpUFBD1+JBwjXoy2bxgZlKo5x98dtAdME46Z4XejpnI5HuyyPbMXWUyBiq6PCDhnkBYnOdYq7zRKGb0V+Zv9ZVkLzG4GMsI7BQLfN2oFXpB+H9n6kc29uE5kvHr04KN6pdNRR/KF6mqJ9rOW47NfDfVi8a9blMblwPYGwHJO6fdYD83iSaXTy9xDbG4Sh6HdbRSY+FEol6OY1gU=
      
      host;x-amz-content-sha256;x-amz-date;x-amz-security-token
      e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, SignatureProvided=d8d0e8513b4573a7c4649dc62d5de8f266f099279ff9f579e05e68bbf920beb5, StringToSign=AWS4-HMAC-SHA256
      20180405T121156Z
      20180405/us-east-1/s3/aws4_request
      72caae579122f71b2ac3be3372c355e51e4aac543ac4e13c8df8f7f1f532bfab, AWSAccessKeyId=ASIAIQFO6AROVPQSLCVQ, StringToSignBytes=41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 38 30 34 30 35 54 31 32 31 31 35 36 5a 0a 32 30 31 38 30 34 30 35 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 37 32 63 61 61 65 35 37 39 31 32 32 66 37 31 62 32 61 63 33 62 65 33 33 37 32 63 33 35 35 65 35 31 65 34 61 61 63 35 34 33 61 63 34 65 31 33 63 38 64 66 38 66 37 66 31 66 35 33 32 62 66 61 62, HostId=8Qk2cHXOIOT1KtGsehcbhhli5N4DyS/G0UyK06aG1FSo6cdby/6pOZdeY2TKgZ+gqcsr//to5Og=}'}
      at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
      ... 42 more
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              carlos Carlos Sanchez
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: