Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features
  3. HADOOP-13611

FileSystem/s3a processDeleteOnExit to skip the exists() check

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Trivial
    • Resolution: Unresolved
    • Affects Version/s: 2.7.3
    • Fix Version/s: None
    • Component/s: fs, fs/s3
    • Labels:
      None
    • Target Version/s:

      Description

      If you look at FileSystem.processDeleteOnExit(), it does an exists() check for each entry, before calling delete().

      That exists() check is superfluous; on s3 it add an extra 1-4 HTTP GETs

      This could be fixed with a subclass in s3a to avoid it, but as the call is superfluous in all filesystems, it could be removed in FileSystem and so picked up by all object stores.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: