Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-429

Unicode filenames on Python 2 cause urlquote to fail

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.2
    • Fix Version/s: 0.14.0-beta3
    • Component/s: Storage
    • Labels:
      None
    • Environment:

      Python 2.6, 2.7

      Description

      Using filenames (or anything that urlquote is passed) on Python 2 cause an exception:

        File "/usr/lib/python2.7/urllib.py", line 1268, in quote
          return ''.join(map(quoter, s))
      KeyError: u'\xe9'
      

      There is a compatibility module libcloud.utils.py3 which is supposed to handle compatibility issues between the two Python versions.

      I've written a test case for this error and a patch. It appears to function correctly on Python 2.6, 2.7 and 3.3.

        Activity

        Hide
        kami Tomaz Muraus added a comment -

        Just a couple of more things:

        1. I've reverted to a different approach (https://git-wip-us.apache.org/repos/asf?p=libcloud.git;a=commitdiff;h=4b755a8bc089a74907ab0efea70d3d41dd1072f5) which also works under Python 2.5 because unicode_literals is not available there and it broke our build.

        2. There were some lint issues, next time please run tox -e lint

        Show
        kami Tomaz Muraus added a comment - Just a couple of more things: 1. I've reverted to a different approach ( https://git-wip-us.apache.org/repos/asf?p=libcloud.git;a=commitdiff;h=4b755a8bc089a74907ab0efea70d3d41dd1072f5 ) which also works under Python 2.5 because unicode_literals is not available there and it broke our build. 2. There were some lint issues, next time please run tox -e lint
        Hide
        kami Tomaz Muraus added a comment -

        I've merged latest patch into trunk.

        Thanks!

        Show
        kami Tomaz Muraus added a comment - I've merged latest patch into trunk. Thanks!
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 3e012e37565c8b8a0d62fc8265d82b2c3b4e9744 in branch refs/heads/trunk from Michael Farrell
        [ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=3e012e3 ]

        LIBCLOUD-429 Fix libcloud.utils.py3.urlquote so it works with unicode
        strings under Python 2.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 3e012e37565c8b8a0d62fc8265d82b2c3b4e9744 in branch refs/heads/trunk from Michael Farrell [ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=3e012e3 ] LIBCLOUD-429 Fix libcloud.utils.py3.urlquote so it works with unicode strings under Python 2.
        Hide
        micolous Michael Farrell added a comment - - edited

        I've extended the unit tests now to include some different tests. These still appear to check out. I've had to wrap the output in b() as urlquote on py3.3 returns a unicode (not bytestring). I think it was related to me previously using a pre-3.3 Mercurial version of Python when I was writing some other patch, and now I'm on 3.3.2.

        https://github.com/Caramel/libcloud/compare/trunk...libcloud-429

        Show
        micolous Michael Farrell added a comment - - edited I've extended the unit tests now to include some different tests. These still appear to check out. I've had to wrap the output in b() as urlquote on py3.3 returns a unicode (not bytestring ). I think it was related to me previously using a pre-3.3 Mercurial version of Python when I was writing some other patch, and now I'm on 3.3.2. https://github.com/Caramel/libcloud/compare/trunk...libcloud-429
        Hide
        kami Tomaz Muraus added a comment -

        The change looks good, but it needs some more tests which test calling this new method with different values (str, unicode, unicode with unicode characters, ...).

        Show
        kami Tomaz Muraus added a comment - The change looks good, but it needs some more tests which test calling this new method with different values (str, unicode, unicode with unicode characters, ...).
        Hide
        micolous Michael Farrell added a comment -

        Unit test that exposes the issue on Python 2.6/2.7:

        https://github.com/Caramel/libcloud/commit/f8e5633e14a420f8ad19b9540ca173a27ba0d2bd

        Fix for the bug, which I've tested and appears to work on Python 2.6/2.7/3.3:

        https://github.com/Caramel/libcloud/commit/ed7d79e50e9a645b435984e774e0ea91f18cab52

        Show
        micolous Michael Farrell added a comment - Unit test that exposes the issue on Python 2.6/2.7: https://github.com/Caramel/libcloud/commit/f8e5633e14a420f8ad19b9540ca173a27ba0d2bd Fix for the bug, which I've tested and appears to work on Python 2.6/2.7/3.3: https://github.com/Caramel/libcloud/commit/ed7d79e50e9a645b435984e774e0ea91f18cab52

          People

          • Assignee:
            Unassigned
            Reporter:
            micolous Michael Farrell
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development