Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11694 Über-jira: S3a phase II: robustness, scale and performance
  3. HADOOP-11874

s3a can throw spurious IOEs on close()

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.7.0
    • None
    • fs/s3
    • None

    Description

      from a code review, it's clear that the issue seen in HADOOP-11851 can surface in S3a, though with HADOOP-11570, it's less likely. It will only happen on those cases when abort() isn't called.

      The "clean" close() code path needs to catch IOEs from the wrappedStream and call abort() in that situation too.

      Attachments

        Issue Links

          Is contained by

          Sub-task - The sub-task of the issue HADOOP-12444 Support lazy seek in S3AInputStream

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-11570 S3AInputStream.close() downloads the remaining bytes of the object from S3

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-11851 s3n to swallow IOEs on inner stream close

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: