Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-725

Binary data storage in LocalStorageDriver fails on Py3

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Storage
    • Labels:
      None
    • Environment:

      LibCloud 0.17.0, Python 3.4

      Description

      LocalStorageDriver.upload_object_via_stream does not allow the upload of bytes in Python 3, despite the fact that download_object_as_stream returns a stream of bytes:

      >>> container.upload_object_via_stream(io.BytesIO(b'foobar'), 'key')
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File
      "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/base.py",
      line 157, in upload_object_via_stream
          iterator, self, object_name, extra=extra, **kwargs)
        File
      "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py",
      line 497, in upload_object_via_stream
          obj_file.close()
        File
      "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py",
      line 69, in __exit__
          raise value
        File
      "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py",
      line 495, in upload_object_via_stream
          obj_file.write(data)
      TypeError: must be str, not bytes
      

      The problem is that LocalStorageDriver opens the target file for writing in text mode ('w') instead of binary mode ('wb').

      As discussed on the users mailing list I am working on a pull request for this issue.

        Activity

        Hide
        torf torf added a comment -

        Fix has been merged.

        Show
        torf torf added a comment - Fix has been merged.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/libcloud/pull/568

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/libcloud/pull/568
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user torfuspolymorphus opened a pull request:

        https://github.com/apache/libcloud/pull/568

        LIBCLOUD-725 Fix handling of binary data in LocalStorageDriver on Py3.

        This PR fixes issue LIBCLOUD-725(https://issues.apache.org/jira/browse/LIBCLOUD-725): `libcloud.storage.driver.local.LocalStorageDriver` has problems with binary data on Python 3.

        The fix is to make sure that files are opened in binary mode for reading and writing.

        Tests and documentation have been updated.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/torfuspolymorphus/libcloud issue725

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/libcloud/pull/568.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #568


        commit cfd7469518072bc59917730267256df78d709d37
        Author: Torf <mail@torf.cc>
        Date: 2015-08-28T10:36:19Z

        LIBCLOUD-725 Fix handling of binary data in LocalStorageDriver on Py3.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user torfuspolymorphus opened a pull request: https://github.com/apache/libcloud/pull/568 LIBCLOUD-725 Fix handling of binary data in LocalStorageDriver on Py3. This PR fixes issue LIBCLOUD-725 ( https://issues.apache.org/jira/browse/LIBCLOUD-725): `libcloud.storage.driver.local.LocalStorageDriver` has problems with binary data on Python 3. The fix is to make sure that files are opened in binary mode for reading and writing. Tests and documentation have been updated. You can merge this pull request into a Git repository by running: $ git pull https://github.com/torfuspolymorphus/libcloud issue725 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/libcloud/pull/568.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #568 commit cfd7469518072bc59917730267256df78d709d37 Author: Torf <mail@torf.cc> Date: 2015-08-28T10:36:19Z LIBCLOUD-725 Fix handling of binary data in LocalStorageDriver on Py3.

          People

          • Assignee:
            Unassigned
            Reporter:
            torf torf
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development