Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1263

Listing objects in Openstack Swift blob store containers fails if a key has an invalid URL encode sequence

    Details

      Description

      If an object is PUT to a container that has an invalid URL encode sequence, subsequence calls to list objects in that container fail due to an attempt to double-URLDecode the object name.

      A small project that reproduces the issue against the latest Devstack Swift can be found here: https://github.com/UniversityofWarwick/jclouds-urlencode-bug

      In short:

      >> PUT http://137.205.194.8:8080/v1/AUTH_d92af3a8d7f44a1091fe38f863ebb69e/encode-test/Files/OpenOffice.org%203.3%20%28en-GB%29%20Installation%20Files/openofficeorg1%25%24.cab HTTP/1.1
      << HTTP/1.1 201 Created
      
      >> GET http://137.205.194.8:8080/v1/AUTH_d92af3a8d7f44a1091fe38f863ebb69e/encode-test?format=json&prefix=Files/ HTTP/1.1
      << HTTP/1.1 200 OK
      << "[{"hash": "0cbc6611f5540bd0809a388dc95a615b", "last_modified": "2017-03-31T09:34:41.522970", "bytes": 4, "name": "Files/OpenOffice.org 3.3 (en-GB) Installation Files/openofficeorg1%$.cab", "content_type": "application/unknown"}]"
      

      In the JSON that Swift returns the key isn't URL encoded Files/OpenOffice.org 3.3 (en-GB) Installation Files/openofficeorg1%$.cab, but when JClouds parses the response in ParseObjectListFromResponse$ToSwiftObject, it builds a URI on the assumption that the path is URL-encoded and this throws an exception due to the invalid escape sequence:

      java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape (%) pattern - For input string: "$."
      
          at java.net.URLDecoder.decode(URLDecoder.java:194)
          at org.jclouds.util.Strings2.urlDecode(Strings2.java:131)
          at org.jclouds.http.Uris$UriBuilder.path(Uris.java:143)
          at org.jclouds.http.Uris$UriBuilder.appendPath(Uris.java:151)
          at org.jclouds.openstack.swift.v1.functions.ParseObjectListFromResponse$ToSwiftObject.apply(ParseObjectListFromResponse.java:99)
          at org.jclouds.openstack.swift.v1.functions.ParseObjectListFromResponse$ToSwiftObject.apply(ParseObjectListFromResponse.java:78)
          at com.google.common.collect.Lists$TransformingRandomAccessList$1.transform(Lists.java:582)
          at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
          at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
          at java.util.AbstractCollection.addAll(AbstractCollection.java:343)
          at com.google.common.collect.Iterables.addAll(Iterables.java:348)
          at org.jclouds.blobstore.domain.internal.PageSetImpl.<init>(PageSetImpl.java:31)
          at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.list(RegionScopedSwiftBlobStore.java:254)
          at uk.ac.warwick.urlencoding.AbstractJCloudsURLEncodingTest.assertCanPutAndList(AbstractJCloudsURLEncodingTest.java:62)
          at uk.ac.warwick.urlencoding.AbstractJCloudsURLEncodingTest.notEncodedWithIllegalEncode(AbstractJCloudsURLEncodingTest.java:102)
          at uk.ac.warwick.urlencoding.SwiftJCloudsURLEncodingTest.notEncodedWithIllegalEncode(SwiftJCloudsURLEncodingTest.java:7)
          ...
      

        Attachments

          Activity

            People

            • Assignee:
              gaul Andrew Gaul
              Reporter:
              mmannion Mat Mannion
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: