[HADOOP-18741] AWS SDK v2 code tuning

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 3.4.0
• Component/s: fs/s3
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 15/May/23 14:24
• Updated: 23/Aug/23 10:35
• Resolved: 23/Aug/23 10:35

Description

tuning of the v2 sdk code prior to merge;

* 

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/AwsCredentialListProvider.java
L184 access denied exception. add test for this?

AWSClientConfig
TODO: Don't think you can set a socket factory for the netty client.


cloudstore: add the new paths
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.thirdparty.org.apache.http.conn.ssl.SSLConnectionSocketFactory;  
oftware.amazon.awssdk.services.s3.model.HeadBucketResponse;

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/HeaderProcessing.java
+add test for getHeaders(/) to see what comes back

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3ABucketExistence.java
L128 use explicit region constant rather than inline string

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AConfiguration.java
L552: use intercept()

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEndpointRegion.java
L75: just throw the exception again
L87, L90, use constants

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java
L44 move o.a.h. imports into "real" hadoop block; include the sets one too

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AProxy.java
is new ssl.proxy  setting consistent with what this pr does

hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationInFileystem.java
L335 TODO open, getObjectMetadata("/")

+cut hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/InconsistentS3ClientFactory.java

Issue Links

• Is contained by: HADOOP-18820 AWS SDK v2: make the v1 bridging support optional