[LIBCLOUD-78] Implement "LazyList" class which implements iterator interface and encapsulates paging

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5.2
• Component/s: Storage
• Labels: "lazy iterator list paging
• Assignee: Unassigned
• Reporter: Tomaz Muraus
• Created: 11/Apr/11 00:19
• Updated: 07/Nov/12 11:25
• Resolved: 26/Jun/11 10:27

Description

Develop a generic LazyList class which implements iterator interface and can be used with the CloudFiles and S3 driver.

CloudFiles and S3 API limits the maximum number of objects which can be returned in a single response which means that on subsequent request you need to provider extra parameters to get the next set of results (marker, prefix, etc.).

This class should transparently encapsulate this paging process and list_container_objects() in CloudFiles and S3 method should return an instance of this class.

The class should also be "lazy" which means that new set of results should only be requested when the current result set is exhausted.

Issue Links

• relates to: LIBCLOUD-254 Provide generator based iteration instead of LazyList