[HDDS-742] Handle object list requests (GET bucket) without prefix parameter

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: S3
• Target Version/s: 0.3.0
• Assignee: Marton Elek
• Reporter: Marton Elek
• Created: 26/Oct/18 12:40
• Updated: 30/Oct/18 22:41
• Resolved: 30/Oct/18 22:41

Description

In s3 gateway the GET bucket endpoint is already implemented. It can return with the available objects based on a given prefix.

(https://docs.aws.amazon.com/AmazonS3/latest/API/v2-RESTBucketGET.html)

As it's defined the Delimiter parameter is used to reduce the response with returning only the first-level keys and prefixes (aka directories)

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Name>example-bucket</Name>
  <Prefix></Prefix>
  <KeyCount>2</KeyCount>
  <MaxKeys>1000</MaxKeys>
  <Delimiter>/</Delimiter>
  <IsTruncated>false</IsTruncated>
  <Contents>
    <Key>sample.jpg</Key>
    <LastModified>2011-02-26T01:56:20.000Z</LastModified>
    <ETag>&quot;bf1d737a4d46a19f3bced6905cc8b902&quot;</ETag>
    <Size>142863</Size>
    <StorageClass>STANDARD</StorageClass>
  </Contents>
  <CommonPrefixes>
    <Prefix>photos/</Prefix>
  </CommonPrefixes>
</ListBucketResult>

Here we can have multiple additional objects with photos/ prefix but they are not added to the response.

 

The main problem in the ozone s3 implementation is that the Delimiter parameter should be optional. In case of the delimiter is missing we should always return all the keys without any common prefix simplification.

It requires for recursive directory listing which is used by s3a adapter.

Issue Links

• duplicates: HDDS-659 Implement pagination in GET bucket (object list) endpoint