Uploaded image for project: 'Hadoop Distributed Data Store'
  1. Hadoop Distributed Data Store
  2. HDDS-434 Provide an s3 compatible REST api for ozone objects
  3. HDDS-742

Handle object list requests (GET bucket) without prefix parameter

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: S3
    • Labels:
      None
    • Target Version/s:

      Description

      In s3 gateway the GET bucket endpoint is already implemented. It can return with the available objects based on a given prefix.

      (https://docs.aws.amazon.com/AmazonS3/latest/API/v2-RESTBucketGET.html)

      As it's defined the Delimiter parameter is used to reduce the response with returning only the first-level keys and prefixes (aka directories)

      <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
        <Name>example-bucket</Name>
        <Prefix></Prefix>
        <KeyCount>2</KeyCount>
        <MaxKeys>1000</MaxKeys>
        <Delimiter>/</Delimiter>
        <IsTruncated>false</IsTruncated>
        <Contents>
          <Key>sample.jpg</Key>
          <LastModified>2011-02-26T01:56:20.000Z</LastModified>
          <ETag>&quot;bf1d737a4d46a19f3bced6905cc8b902&quot;</ETag>
          <Size>142863</Size>
          <StorageClass>STANDARD</StorageClass>
        </Contents>
        <CommonPrefixes>
          <Prefix>photos/</Prefix>
        </CommonPrefixes>
      </ListBucketResult>

      Here we can have multiple additional objects with photos/ prefix but they are not added to the response.

       

      The main problem in the ozone s3 implementation is that the Delimiter parameter should be optional. In case of the delimiter is missing we should always return all the keys without any common prefix simplification.

      It requires for recursive directory listing which is used by s3a adapter.

        Attachments

        1. HDDS-742.001.patch
          3 kB
          Elek, Marton
        2. HDDS-742.02.patch
          6 kB
          Bharat Viswanadham

          Issue Links

            Activity

              People

              • Assignee:
                elek Elek, Marton
                Reporter:
                elek Elek, Marton
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: