Libcloud
  1. Libcloud
  2. LIBCLOUD-233

Atmos storage driver doesn't correctly encode path names

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.11.0
    • Component/s: Storage
    • Labels:
      None
    • Environment:

      Python 2.7.1

      Description

      If you use the Atmos storage driver, and you attempt to stream the upload of an object, and either your container name or your object name is a unicode string, the presence of these unicode strings will cause the HTTP message body to be converted into a unicode string.

      However, file content is provided as a byte string; if the file content contains binary data, httplib will try to convert this file content into unicode, yielding encoding errors.

      For example, if you try to stream upload a PDF whose name is stored as u'foo.pdf', you'll get a message something like:

      UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 10: ordinal not in range(128)

      (Position 10 is where the binary content in a PDF starts, after the "%PDF-1.3\n%" header)

      The behaviour of httplib in the presence of unicode content is a known issue (http://bugs.python.org/issue12398); All path tokens should be encoded as ascii before being passed to httplib to prevent this problem occurring.

      As far as I can make out, this problem only exists under Python 2.7 – I've observed it on Python 2.7.1 and Python 2.7.3. Python 2.6.7 is unaffected.

        Activity

        Hide
        Tomaz Muraus added a comment -

        Merged into trunk, thanks.

        Show
        Tomaz Muraus added a comment - Merged into trunk, thanks.
        Hide
        Russell Keith-Magee added a comment -

        Patch provided as part of pull request https://github.com/apache/libcloud/pull/70

        Show
        Russell Keith-Magee added a comment - Patch provided as part of pull request https://github.com/apache/libcloud/pull/70

          People

          • Assignee:
            Tomaz Muraus
            Reporter:
            Russell Keith-Magee
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development