[LIBCLOUD-823] UnicodeDecodeError when uploading via Azure Blobs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Storage

Environment

Ubuntu 14.04.3
Python 2.7.6
apache-libcloud 0.20.1

• Assignee: Unassigned
• Reporter: Jamie Cressey
• Created: 26/May/16 17:18
• Updated: 26/May/16 17:43
• Resolved: 26/May/16 17:43

Description

I take regular mongo backups which are compressed as tgz files and then uploaded to the storage platform of the cloud provider hosting the DB servers, either S3 and Azure Blobs. The following code works fine for S3:

from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver

cls = get_driver(Provider.S3_EU_WEST)
driver = cls(cfg['storage']['user'],
             cfg['storage']['pass'])

try:
        container = driver.create_container(cfg['storage']['container'])
except:
        container = driver.get_container(cfg['storage']['container'])

driver.upload_object(
        '/mnt/resource/backup/full_20160526_080013.tgz',
        container,
        'mongodb/full_20160526_080013.tgz',
)

However, running this against Azure Blobs, returns the following error:

Traceback (most recent call last):
  File "/usr/local/scripts/mongodb_backup.py", line 645, in main
    upload_local(backup_root, backup_dir, ml, mc, sp, container)
  File "/usr/local/scripts/mongodb_backup.py", line 156, in upload_local
    verify_hash=True):
  File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/drivers/azure_blobs.py", line 795, in upload_object
    use_lease=ex_use_lease)
  File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/drivers/azure_blobs.py", line 932, in _put_object
    iterator=iterator)
  File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/base.py", line 654, in _upload_object
    **upload_func_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/libcloud/storage/drivers/azure_blobs.py", line 664, in _upload_in_chunks
    params=params)
  File "/usr/local/lib/python2.7/dist-packages/libcloud/common/base.py", line 782, in request
    headers=headers)
  File "/usr/lib/python2.7/httplib.py", line 979, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1013, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 975, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 833, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128)