Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13204 Über-jira: S3a phase III: scale and tuning
  3. HADOOP-13712

S3A open to avoid needless HEAD on the successful execution path

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 2.7.3
    • None
    • fs/s3
    • None

    Description

      S3A's open() operation does a getFileStatus() check to see if a file is not a directory before opening with a GET. That initial check will take up at least one HEAD request if the file is present, more if it isn't.

      As the GET itself performs the existence check, it is needless. A successful GET of a path which doesn't end in "/" means a file was there. The only reason a getFileStatus call is needed is to choose which error message to display if the path isn't there: is it an FNFE or is it path-is-directory.

      Proposed: reorder the code to do the GET; only if that fails fallback to getFileStatus()

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment