[HADOOP-17451] Intermittent failure of S3A tests which make assertions on statistics/IOStatistics

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.3.1
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 31/Dec/20 21:59
• Updated: 02/Feb/23 12:53
• Resolved: 12/Jan/21 17:25

Description

Intermittent failure of ITestHuge* upload tests, when doing parallel test runs.

The count of bytes uploaded through StorageStatistics isn't updated. Maybe the expected counter isn't updated, and somehow in a parallel run with recycled FS instances/set up directory structure this surfaces the way it doesn't in a single test run.

Issue Links

• causes: HADOOP-18611 AbstractSTestS3AHugeFiles put request assert is using wrong iostatistic
• contains: HADOOP-17455 [s3a] Intermittent failure of ITestS3ADeleteCost.testDeleteSingleFileInDir
• contains: HADOOP-17456 S3A ITestPartialRenamesDeletes.testPartialDirDelete[bulk-delete=true] failure
• is broken by: HADOOP-17271 S3A statistics to support IOStatistics
• links to: GitHub Pull Request #2594