Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17566 Über-jira: S3A Hadoop 3.3.2 features
  3. HADOOP-17774

bytesRead FS statistic showing twice the correct value in S3A

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.2
    • fs/s3

    Description

      S3A "bytes read" statistic is being incremented twice. Firstly while reading in S3AInputStream and then in merge() of S3AInstrumentation when S3AInputStream is closed.

      This makes "bytes read" statistic equal to sum of stream_read_bytes and stream_read_total_bytes.

      Attachments

        Issue Links

          is caused by

          Sub-task - The sub-task of the issue HADOOP-17271 S3A statistics to support IOStatistics

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-17789 S3 CSV read performance with Spark with Hadoop 3.3.1 is slower than older Hadoop

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link GitHub Pull Request #3144

          Activity

            People

              mehakmeetSingh Mehakmeet Singh
              mehakmeetSingh Mehakmeet Singh
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2.5h
                  2.5h