Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11694 Über-jira: S3a phase II: robustness, scale and performance
  3. HADOOP-13871

ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0, 2.9.0, 3.0.0-alpha2
    • 2.8.0, 3.0.0-alpha2
    • fs/s3
    • None
    • landsat bucket from the UK

    • Reviewed

    Description

      The ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks takes 15s on branch-2, but is now taking minutes.

      This is a regression, and it's surfacing on some internal branches too. Even where the code hasn't changed. It does not happen on branch-2, which has a later SDK.

      Attachments

        1. org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance-output.txt
          1.10 MB
          Steve Loughran
        2. HADOOP-13871-001.patch
          2 kB
          Steve Loughran
        3. HADOOP-13871-002.patch
          11 kB
          Steve Loughran
        4. HADOOP-13871-003.patch
          12 kB
          Steve Loughran

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: