[HADOOP-19210] s3a: TestS3AAWSCredentialsProvider and TestS3AInputStreamRetry really slow

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.5.0
• Fix Version/s: 3.5.0, 3.4.1
• Component/s: fs/s3, test
• Labels: pull-request-available
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 26/Jun/24 18:24
• Updated: 02/Jul/24 17:02
• Resolved: 02/Jul/24 10:36

Description

Not noticed this before, but the unit tests TestS3AAWSCredentialsProvider and TestS3AInputStreamRetry are so slow they will be hurting over all test performance times: no integration tests will start until these are all complete.

mvn test -T 1C -Dparallel-tests

...
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.877 s - in org.apache.hadoop.fs.s3a.TestS3AInputStreamRetry
...
[INFO] Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 90.038 s - in org.apache.hadoop.fs.s3a.TestS3AAWSCredentialsProvider

The PR cuts total execution time of a 10 thread test run from 3 minutes to 2:30

Issue Links

• links to: GitHub Pull Request #6907