[SPARK-24000] S3A: Create Table should fail on invalid AK/SK

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Done
• Affects Version/s: 2.3.0
• Component/s: Spark Shell
• Assignee: Unassigned
• Reporter: Brahma Reddy Battula
• Created: 17/Apr/18 13:05
• Updated: 21/Feb/20 14:03
• Resolved: 21/Feb/20 14:03

Description

Currently, When we pass the invalid ak&&sk create table will be the success.

when the S3AFileSystem is initialized, verifyBucketExists() is called, which will return True as the status code 403 (BUCKET_ACCESS_FORBIDDEN_STATUS_CODE)  from following as bucket exists.

public boolean doesBucketExist(String bucketName)
         throws AmazonClientException, AmazonServiceException {
 
     try {
         headBucket(new HeadBucketRequest(bucketName));
         return true;
     } catch (AmazonServiceException ase) {
         // A redirect error or a forbidden error means the bucket exists. So
         // returning true.
         if ((ase.getStatusCode() == Constants.BUCKET_REDIRECT_STATUS_CODE)
                 || (ase.getStatusCode() == Constants.BUCKET_ACCESS_FORBIDDEN_STATUS_CODE)) {
             return true;
         }
         if (ase.getStatusCode() == Constants.NO_SUCH_BUCKET_STATUS_CODE) {
             return false;
         }
         throw ase;
 
     }
 }

Issue Links

• depends upon: HADOOP-16711 S3A bucket existence checks to support v2 API and "no checks at all"
• is related to: HADOOP-15990 S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2
• is related to: HADOOP-15409 S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2