Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18477 Über-jira: S3A Hadoop 3.3.9 features
  3. HADOOP-14837

Handle S3A "glacier" data

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.0.0-beta1
    • None
    • fs/s3

    Description

      SPARK-21797 covers how if you have AWS S3 set to copy some files to glacier, they appear in the listing but GETs fail, and so does everything else

      We should think about how best to handle this.

      1. report better
      2. if listings can identify files which are glaciated then maybe we could have an option to filter them out
      3. test & see what happens

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HADOOP-12020 Support configuration of different S3 storage classes

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-16259 Distcp to set S3 Storage Class

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-21797 spark cannot read partitioned data in S3 that are partly in glacier

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-16077 Add an option in ls command to include storage policy

          • Major - Major loss of function.
          • Patch Available
          links to

          Web Link GitHub Pull Request #6407

          Activity

            People

              bpahuja Bhavay Pahuja
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated: