If you look at FileSystem.processDeleteOnExit(), it does an exists() check for each entry, before calling delete().
That exists() check is superfluous; on s3 it add an extra 1-4 HTTP GETs
This could be fixed with a subclass in s3a to avoid it, but as the call is superfluous in all filesystems, it could be removed in FileSystem and so picked up by all object stores.
- is duplicated by
-
HADOOP-16877 S3A FS deleteOnExit to skip the exists check
-
- Resolved
-
- links to