Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1475

Exception listing S3 dir where path contains %

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.1
    • Fix Version/s: None
    • Component/s: jclouds-core
    • Labels:
      None
    • Environment:
      MacBook Pro, OSX High Sierra (10.13.6)

      Description

      Issue 278 was a similar problem, seemingly pretty different code path.

      Doing S3BlobStore.list, if directory name is something like Baz%, an exception is thrown. Encoding the % avoids the exception, but does not find the directory to return results. Interestingly, %Baz works. I believe the problem is that http.utils.Queries.parseQueryToDecodedMap decodes the entire path string, when it really only cases about the parameters. I think %Baz works because %Ba is a valid escape pattern. So it gets put in the parameter map, but never used.

      Here's a call stack. My path within an S3 bucket is "dgb6/Foo/Baz%" .

      ERROR CloudStoreManager 2018-12-14 11:53:57,546 http-nio-8080-exec-1 : Failed to delete directory dgb6/Foo/Baz%
      java.lang.IllegalArgumentException: URLDecoder: Incomplete trailing escape (%) pattern
       at java.net.URLDecoder.decode(URLDecoder.java:187)
       at org.jclouds.util.Strings2.urlDecode(Strings2.java:131)
       at org.jclouds.http.utils.Queries.parseKeyValueFromStringToDecodedMap(Queries.java:61)
       at org.jclouds.http.utils.Queries.access$000(Queries.java:30)
       at org.jclouds.http.utils.Queries$1.apply(Queries.java:45)
       at org.jclouds.http.utils.Queries$1.apply(Queries.java:32)
       at org.jclouds.aws.s3.filters.AWSRequestAuthorizeSignatureV4.signForAuthorizationHeader(AWSRequestAuthorizeSignatureV4.java:56)
       at org.jclouds.s3.filters.RequestAuthorizeSignatureV4.filter(RequestAuthorizeSignatureV4.java:55)
       at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:90)
       at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
       at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
       at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
       at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
       at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
       at com.sun.proxy.$Proxy79.listBucket(Unknown Source)
       at org.jclouds.s3.blobstore.S3BlobStore.list(S3BlobStore.java:177)
       at org.labkey.cloud.store.model.CloudStoreManager.clearContainer(CloudStoreManager.java:1047)
      

       

      Here's the code:

      ListContainerOptions listOpts = ListContainerOptions.Builder.prefix(dir.toString());
      for (StorageMetadata meta : store.list(bucket, listOpts))
      {
       if (!dirString.equals(meta.getName()) &&
       (StorageType.FOLDER.equals(meta.getType()) || StorageType.RELATIVE_PATH.equals(meta.getType())))
       {
       clearContainer(store, bucket, new Path(meta.getName()));
       }
      }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              labkey-dave Dave Bradlee
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: