[LIBCLOUD-993] upload_object_via_stream crashes for Azure driver

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.4.0
• Component/s: Storage
• Labels: azure
• Assignee: Unassigned
• Reporter: Clemens Wolff
• Created: 17/Apr/18 20:27
• Updated: 30/May/19 18:32
• Resolved: 30/May/19 18:24

Description

Running the examples in the documentation for object upload via streams fails when using the Azure Blob Storage driver.

Setup:

python3 -m venv venv
venv/bin/pip install apache-libcloud==2.3.0

Minimal stand-alone code snippet to reproduce:

import uuid
import os

from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver

key = ''  # setme
secret = ''  # setme
container_name = 'libcloudtest-%s' % uuid.uuid4()
file_path = __file__
object_name = os.path.basename(__file__)

cls = get_driver(Provider.AZURE_BLOBS)
driver = cls(key, secret)

container = driver.create_container(container_name)

with open(file_path, 'rb') as fobj:
    driver.upload_object_via_stream(iterator=fobj,
                                    container=container,
                                    object_name=object_name)

Stack-trace:

Traceback (most recent call last):
File "./azure_upload.py", line 24, in <module>
driver.upload_object_via_stream(iterator=fobj, container=container, object_name=object_name)
File "/c/Repos/libcloud/libcloud/storage/drivers/azure_blobs.py", line 821, in upload_object_via_stream
stream=iterator)
File "/c/Repos/libcloud/libcloud/storage/drivers/azure_blobs.py", line 924, in _put_object
driver=self)
libcloud.common.types.LibcloudError: <LibcloudError in <libcloud.storage.drivers.azure_blobs.AzureBlobsStorageDriver object at 0x7ff3e2132400> 'Unexpected status code, status_code=403'>

Note that the culprit is the upload_object_via_stream method. Running the same snippet with upload_object instead of the stream method works:

import uuid
import os

from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver

key = ''  # setme
secret = ''  # setme
container_name = 'libcloudtest-%s' % uuid.uuid4()
file_path = __file__
object_name = os.path.basename(__file__)

cls = get_driver(Provider.AZURE_BLOBS)
driver = cls(key, secret)

container = driver.create_container(container_name)

driver.upload_object(file_path=file_path,
                     container=container,
                     object_name=object_name)

 If you need Azure credentials to reproduce or debug this issue, feel free to reach out and I can provide them.