After getting in
IMPALA-4172/ IMPALA-3653, we expected good gains in S3 metadata loading. However, as Mostafa Mokhtar found recently, we spend a lot of time sending requests to S3 for every file to check the ACL permissions associated with it.
However, on S3, there are no Hadoop style ACLs but only object-level ACLs which are associated with the AWS credentials accessing the files (objects) and we cannot yet check or set permissions with the S3AFileSystem.
This means that we're wasting a lot of time waiting for S3 only to get back the same standard response every time. We can skip these checks when we're working with S3 files just the same way we skip 'inherit permissions' while creating new S3 files.