Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16829 Über-jira: S3A Hadoop 3.4 features
  3. HADOOP-17134

S3AFileSystem.listLocatedStatu(file) does a LIST even with S3Guard

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.4.0
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None

      Description

      This is minor and we may want to WONTFIX; noticed during work on directory markers.

      If you call listLocatedStatus(file) then a LIST call is always made to S3, even when S3Guard is present and has the record to say "this is a file"

      Does this matter enough to fix?

      1. The HADOOP-16465 work moved the list before falling back to getFileStatus
      2. that listing calls s3guard.listChildren(path) to list the children.
      3. which only returns the chlldren of a path, not a record of the path itself.
      4. so we get an empty list back, triggering the LIST
      5. its only after that LIST fails that we fall back to getFileStatus and hence look for the actual file record.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mukund-thakur Mukund Thakur
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: