Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-429

Unicode filenames on Python 2 cause urlquote to fail

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 0.13.2
        • Fix Version/s: 0.14.0-beta3
        • Component/s: Storage
        • Labels:
          None
        • Environment:

          Python 2.6, 2.7

          Description

          Using filenames (or anything that urlquote is passed) on Python 2 cause an exception:

            File "/usr/lib/python2.7/urllib.py", line 1268, in quote
    return ''.join(map(quoter, s))
KeyError: u'\xe9'

          There is a compatibility module libcloud.utils.py3 which is supposed to handle compatibility issues between the two Python versions.

          I've written a test case for this error and a patch. It appears to function correctly on Python 2.6, 2.7 and 3.3.

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            micolous Michael Farrell added a comment -

            Unit test that exposes the issue on Python 2.6/2.7:

            https://github.com/Caramel/libcloud/commit/f8e5633e14a420f8ad19b9540ca173a27ba0d2bd

            Fix for the bug, which I've tested and appears to work on Python 2.6/2.7/3.3:

            https://github.com/Caramel/libcloud/commit/ed7d79e50e9a645b435984e774e0ea91f18cab52

            Show
            micolous Michael Farrell added a comment - Unit test that exposes the issue on Python 2.6/2.7: https://github.com/Caramel/libcloud/commit/f8e5633e14a420f8ad19b9540ca173a27ba0d2bd Fix for the bug, which I've tested and appears to work on Python 2.6/2.7/3.3: https://github.com/Caramel/libcloud/commit/ed7d79e50e9a645b435984e774e0ea91f18cab52
            Hide
            Permalink
            kami Tomaz Muraus added a comment -

            The change looks good, but it needs some more tests which test calling this new method with different values (str, unicode, unicode with unicode characters, ...).

            Show
            kami Tomaz Muraus added a comment - The change looks good, but it needs some more tests which test calling this new method with different values (str, unicode, unicode with unicode characters, ...).
            Hide
            Permalink
            micolous Michael Farrell added a comment - - edited

            I've extended the unit tests now to include some different tests. These still appear to check out. I've had to wrap the output in b() as urlquote on py3.3 returns a unicode (not bytestring). I think it was related to me previously using a pre-3.3 Mercurial version of Python when I was writing some other patch, and now I'm on 3.3.2.

            https://github.com/Caramel/libcloud/compare/trunk...libcloud-429

            Show
            micolous Michael Farrell added a comment - - edited I've extended the unit tests now to include some different tests. These still appear to check out. I've had to wrap the output in b() as urlquote on py3.3 returns a unicode (not bytestring ). I think it was related to me previously using a pre-3.3 Mercurial version of Python when I was writing some other patch, and now I'm on 3.3.2. https://github.com/Caramel/libcloud/compare/trunk...libcloud-429
            Hide
            Permalink
            jira-bot ASF subversion and git services added a comment -

            Commit 3e012e37565c8b8a0d62fc8265d82b2c3b4e9744 in branch refs/heads/trunk from Michael Farrell
            [ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=3e012e3 ]

            LIBCLOUD-429 Fix libcloud.utils.py3.urlquote so it works with unicode
            strings under Python 2.

            Show
            jira-bot ASF subversion and git services added a comment - Commit 3e012e37565c8b8a0d62fc8265d82b2c3b4e9744 in branch refs/heads/trunk from Michael Farrell [ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=3e012e3 ] LIBCLOUD-429 Fix libcloud.utils.py3.urlquote so it works with unicode strings under Python 2.
            Hide
            Permalink
            kami Tomaz Muraus added a comment -

            I've merged latest patch into trunk.

            Thanks!

            Show
            kami Tomaz Muraus added a comment - I've merged latest patch into trunk. Thanks!
            Hide
            Permalink
            kami Tomaz Muraus added a comment -

            Just a couple of more things:

            1. I've reverted to a different approach (https://git-wip-us.apache.org/repos/asf?p=libcloud.git;a=commitdiff;h=4b755a8bc089a74907ab0efea70d3d41dd1072f5) which also works under Python 2.5 because unicode_literals is not available there and it broke our build.

            2. There were some lint issues, next time please run tox -e lint

            Show
            kami Tomaz Muraus added a comment - Just a couple of more things: 1. I've reverted to a different approach ( https://git-wip-us.apache.org/repos/asf?p=libcloud.git;a=commitdiff;h=4b755a8bc089a74907ab0efea70d3d41dd1072f5 ) which also works under Python 2.5 because unicode_literals is not available there and it broke our build. 2. There were some lint issues, next time please run tox -e lint

              People

              • Assignee:
                Unassigned
                Reporter:
                micolous Michael Farrell
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development