Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11694 Über-jira: S3a phase II: robustness, scale and performance
  3. HADOOP-13286

add a S3A scale test to do gunzip and linecount

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.8.0
    • None
    • fs/s3
    • None

    Description

      the HADOOP-13203 patch proposal showed that there were performance problems downstream which weren't surfacing in the current scale tests.

      Trying to decompress the .gz test file and then go through it with LineReader models a basic use case: parse a .csv.gz data source.

      Add this, with metric printing

      Attachments

        1. HADOOP-13286-branch-2-001.patch
          5 kB
          Steve Loughran

        Issue Links

          is part of

          Sub-task - The sub-task of the issue HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: