[HADOOP-18072] Über-JIRA: abfs phase III: Hadoop 3.4.0 features & fixes

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Done
• Affects Version/s: 3.3.2
• Fix Version/s: 3.4.0
• Component/s: fs/azure
• Assignee: Anmol Asrani
• Reporter: Steve Loughran
• Created: 07/Jan/22 14:09
• Updated: 27/Feb/24 11:12
• Resolved: 27/Feb/24 11:10

Description

ABFS related changes for hadoop 3.3.3

Sub-Tasks

• ABFS: Stabilize openFile withStatus: Resolved
• ABFS tests to include terasort: Resolved
• ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters: Resolved
• ABFS: Use Unique File Paths in Tests: Resolved
• ABFS: Enable config controlled ETag check for Rename idempotency: Resolved
• ABFS: Support for Encryption Context: Resolved
• ABFS: Set driver global timeout for ITestAzureBlobFileSystemBasics: Resolved
• ABFS: Fix unchecked cast compiler warning for AbfsListStatusRemoteIterator: Resolved
• ABFS: Fix failure caused by listFiles() in ITestAbfsRestOperationException: Resolved
• ABFS: Add testfilePath while verifying test contents are read correctly: Resolved
• ABFS: Add changes for expect hundred continue header with append requests: Resolved
• ABFS: Support for account level throttling: Resolved
• Add reason in in x-ms-client-request-id on a retry API call.: Resolved
• x-ms-client-request-id to have some way that identifies retry of an API.: Resolved
• ABFS: eTag Check to mitigate Rename Failures: Resolved
• ABFS: Fix failing CPK tests on trunk: Resolved
• ABFS: Adding 100 continue in userAgent String and dynamically removing it if retry is without the header enabled.: Resolved
• ABFS: Fixing Behavior of a File System APIs on root path: Resolved
• ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations: Resolved
• ABFS: Adding Server returned request id in Exception method thrown to caller.: Resolved
• ABFS: Change default from disk to bytebuffer for fs.azure.data.blocks.buffer: Resolved
• Expect-100 JDK bug resolution: prevent multiple server calls: Resolved
• ABFS: Adding Support for MD5 Hash based integrity verification of the request content during transport: Resolved
• ABFS: Enable Footer Read Optimizations with Appropriate Footer Read Buffer Size: Resolved