[HADOOP-18231] tests in ITestS3AInputStreamPerformance are failing

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Ahmar Suhail Assign to me
• Reporter: Ahmar Suhail
• Created: 10/May/22 12:04
• Updated: 28/Jan/24 08:23
• Resolved: 15/Jul/22 12:52

Description

The following tests are failing when prefetching is enabled:

testRandomIORandomPolicy - expects stream to be opened 4 times (once for every random read), but prefetching will only open twice. 

testDecompressionSequential128K - expects stream to be opened once, but prefetching will open once for each block the file has. landsat file used in the test has size 42MB, prefetching block size = 8MB, expected open count is 6.

 testReadWithNormalPolicy - same as above. 

testRandomIONormalPolicy - executes random IO, but with a normal policy. S3AInputStream will abort the stream and change the policy, prefetching handles random IO by caching blocks so doesn't do any of that. 

testRandomReadOverBuffer - multiple assertions failing here, also depends a lot on readAhead values, not very relevant for prefetching

Issue Links

• is required by: HADOOP-18243 Rebase s3a prefetching feature branch on top of trunk
• links to: GitHub Pull Request #4305
• links to: GitHub Pull Request #4386