[HUDI-4495] Specific S3 URI patterns break with Hudi

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Labels: pull-request-available release-blocker
• Assignee: Udit Mehrotra
• Reporter: Udit Mehrotra
• Created: 28/Jul/22 21:35
• Updated: 09/Aug/22 16:27
• Resolved: 09/Aug/22 16:27

Description

Current certain S3 path patterns break with Hudi, for ex: paths with <string>.<integer> pattern in S3 bucket name. This error stems from https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/fs/HoodieWrapperFileSystem.java#L148 where it uses getHost() API while converting to new Scheme. While this works well for HDFS where it always has a host, it does not work well with this specific pattern which is a valid S3 bucket name.

For example:

s3://bucket.1234/table1 gets incorrectly converted to s3:/table1

Instead we should use getAuthority() which correctly gets the S3 bucket name as the authority.

Issue Links

• links to: GitHub Pull Request #6237