Description
Currently there are two options for uploading objects to any given storage; StorageDriver.upload_object() and StorageDriver.upload_object_via_stream(). The former strictly requires the object to be written to disk prior to upload since it takes the file path as argument. The latter does support upload given an iterator, but does not work for Amazon S3 since they do not support chunked encoding. After discussing this with Tomaž he and I came to the conclusion that a third method is in order which will take an iterator as argument while not requiring the provider to support chunked encoding.
The solution is to support an iterator as argument and generate a request by iterating through it directly. However, these will require the object to be kept in memory and depending on the size and available resources it might lead to exhaustion. So it should be noted in the methods documentation that the implementation is responsible for preventing such outcomes; which can be done by preventing the allowed upload size for example.