Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17229

Test failure as failed request body counted in byte received metric - ITestAbfsNetworkStatistics#testAbfsHttpResponseStatistics

    XMLWordPrintableJSON

    Details

      Description

      Bytes received counter increments for every request response received. 

      https://github.com/apache/hadoop/blob/d23cc9d85d887f01d72180bdf1af87dfdee15c5a/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java#L251

      This increments even for failed requests. 

      Observed during testing done for HADOOP-17215. A request failed with 409 Conflict contains response body as below:

      {"error":{"code":"PathAlreadyExists","message":"The specified path already exists.\nRequestId:c3b2c55c-b01f-0061-7b31-7b6ee3000000\nTime:2020-08-25T22:44:07.2356054Z"}}

      The error body of 168 size is incremented in bytes_received counter. 

      This also breaks the testcase testAbfsHttpResponseStatistics. 

      [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.746 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.746 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics[ERROR] testAbfsHttpResponseStatistics(org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics)  Time elapsed: 13.183 s  <<< FAILURE!java.lang.AssertionError: Mismatch in bytes_received expected:<143> but was:<311> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:645) at org.apache.hadoop.fs.azurebfs.AbstractAbfsIntegrationTest.assertAbfsStatistics(AbstractAbfsIntegrationTest.java:445) at org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics.testAbfsHttpResponseStatistics(ITestAbfsNetworkStatistics.java:291)
      

      Mehakmeet Singh - is the bytes_received counter increment for failed requests an expected behaviour ?

        Attachments

          Activity

            People

            • Assignee:
              mehakmeetSingh Mehakmeet Singh
              Reporter:
              snvijaya Sneha Vijayarajan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 40m
                2h 40m