[HADOOP-17229] Test failure as failed request body counted in byte received metric - ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.1
• Component/s: fs/azure, test
• Labels: abfsactive pull-request-available
• Assignee: Mehakmeet Singh
• Reporter: Sneha Vijayarajan
• Created: 25/Aug/20 22:11
• Updated: 08/Sep/20 13:14
• Resolved: 08/Sep/20 09:15

Description

Bytes received counter increments for every request response received. 

https://github.com/apache/hadoop/blob/d23cc9d85d887f01d72180bdf1af87dfdee15c5a/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java#L251

This increments even for failed requests. 

Observed during testing done for HADOOP-17215. A request failed with 409 Conflict contains response body as below:

{"error":{"code":"PathAlreadyExists","message":"The specified path already exists.\nRequestId:c3b2c55c-b01f-0061-7b31-7b6ee3000000\nTime:2020-08-25T22:44:07.2356054Z"}}

The error body of 168 size is incremented in bytes_received counter. 

This also breaks the testcase testAbfsHttpResponseStatistics. 

[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.746 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.746 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)  Time elapsed: 13.183 s  <<< FAILURE!java.lang.AssertionError: Mismatch in bytes_received expected:<143> but was:<311> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:645) at org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445) at org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:291)

mehakmeetSingh - is the bytes_received counter increment for failed requests an expected behaviour ?

Issue Links

• links to: GitHub Pull Request #2264
• links to: GitHub Pull Request #2283