Details
-
New Feature
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
Description
While working with libcloud, I thought it will be good to have some utility APIs to containers, which will help in easier use. Some APIs will have a base class implementation, which can be easily over-written in each driver based on the storage provider.
Some APIs that I have in mind are:
1. container.copy_object(obj, target_container)
Copies an object from one container to another. Base class implementation can be
target_container.upload_object_via_stream(container.download_object_as_stream(obj), obj.name)
In each driver, this can be over-written to use specific methods to do this (AWS, Azure etc. provide APIs for doing this).
2. container.move_object(obj, target_container)
Base class implementation can be:
container.copy_object(obj, target_container)
obj.delete()
Again, driver specific extensions can be done.
3. container.sync_to_local(local_path, only_missing=True, only_newer=False)
4. container.sync_from_local(local_path, followlinks=False, prefix='', only_missing=True, only_newer=False)
The above two functions are good for syncing local folder repos with cloud. The functions are ridiculously easy to implement and will provide good mechanism for sync. (For eg: syncing a folder which contains 500 HLS chunks of a video)
The parameters used are:
- only_missing - sync only files that missing in the target (container or folder)
- only_newer - sync only files that have last-modified (or m_time) more than the target
- prefix - A prefix to be prefixed to each object name while syncing from a local folder.
I am just opening a ticket so that we can discuss this and finalize it. Once everyone agree, I can submit/check-in a patch