[HADOOP-19066] AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0, 3.4.1
• Fix Version/s: 3.5.0, 3.4.1
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.5.0, 3.4.1
• Assignee: Viraj Jasani
• Reporter: Viraj Jasani
• Created: 05/Feb/24 06:08
• Updated: 15/Mar/24 12:31
• Resolved: 13/Mar/24 13:31

Description

FIPS support can be enabled by setting "fs.s3a.endpoint.fips". Since the SDK considers overriding endpoint and enabling fips as mutually exclusive, we fail fast if fs.s3a.endpoint is set with fips support (details on HADOOP-18975).

Now, we no longer override SDK endpoint for central endpoint since we enable cross region access (details on HADOOP-19044) but we would still fail fast if endpoint is central and fips is enabled.

Changes proposed:

• S3A to fail fast only if FIPS is enabled and non-central endpoint is configured.
• Tests to ensure S3 bucket is accessible with default region us-east-2 with cross region access (expected with central endpoint).
• Document FIPS support with central endpoint on connecting.html.

Note: there are two patches here on trunk; they've been coalesced into one on branch-3.4.

Issue Links

• is caused by: HADOOP-19044 AWS SDK V2 - Update S3A region logic
• links to: GitHub Pull Request #6539
• links to: GitHub Pull Request #6624