Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-378

S3 uploads fail on small iterators

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.13.1
    • Storage
    • None

    Description

      I wrote a small script that uploaded the output of a buildbot job and then updated an XML file. The large binary blob worked fine. However the XML file failed.

      I was using the driver.upload_object_via_stream(iterator=StringIO.StringIO(somexml)) style as in the docs.

      Looking at the LIBCLOUD_DEBUG output the driver was using the S3 multi-part upload API and making a new "part" for each line - so every 7 bytes or so - but the minimum size for a part upload was 5mb.

      (I don't know if the first part is allowed to be less than 5mb if the entire upload is less than 5mb).

      I am working around this by forcing multi-part uploads off.

      Attachments

        1. LIBCLOUD-378.diff
          0.6 kB
          Mahendra M

        Activity

          People

            mahendra.m Mahendra M
            jc2k John Carr
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment