Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15220

Über-jira: S3a phase V: Hadoop 3.2 features

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      Über-jira for S3A work for Hadoop 3.2.x

      The items from HADOOP-14831 which didn't get into Hadoop-3.1, and anything else

        Attachments

          Issue Links

          1.
          add htrace context to HTTP requests as ? parameter Sub-task Resolved Unassigned
          2.
          S3ADataBlocks.DiskBlock to lazy create dest file for faster 0-byte puts Sub-task Resolved Unassigned
          3.
          use s3 tags/headers to record permissions on objects, so preserving them through distcp round trips Sub-task Resolved Unassigned
          4.
          Automatically determine region & hence S3 endpoint of buckets Sub-task Resolved Unassigned
          5.
          S3A shell entry point to support commands specific to S3A. Sub-task Resolved Unassigned
          6.
          Hive doesn't let s3a patch the credential provider path Sub-task Resolved Unassigned
          7.
          s3a encryption tests fail when dest bucket has SSE-KMS enabled Sub-task Resolved Unassigned
          8.
          AWSCredentialProviderList to throw custom NoCredentialsException; retry logic to handle Sub-task Resolved Unassigned
          9.
          Add Private/Unstable BulkDelete operations to supporting object stores for DistCP Sub-task Resolved Steve Loughran
          10.
          S3AInputStream: Handle http stream skip(n) skipping < n bytes in a forward seek Sub-task Resolved Unassigned
          11.
          S3ABlockOutputStream.flush() be no-op when stream closed Sub-task Resolved Gabor Bota
          12.
          ITestS3AEmptyDirectory.testParallelJobsToAdjacentPaths failure, FileAlreadyExistsException Sub-task Resolved Steve Loughran
          13.
          regression: secret stripping from S3x URIs breaks some downstream code Sub-task Resolved Vinayakumar B
          14.
          S3A ITestS3GuardWriteBack failing if bucket explicitly set to s3guard+DDB Sub-task Resolved Steve Loughran
          15.
          S3 returning 400 on the directory /test/ GET of getFileStatus Sub-task Resolved Steve Loughran
          16.
          S3ARetryPolicy for 400/BadArgument to be "fail" Sub-task Resolved Steve Loughran
          17.
          Stabilize/tune S3A committers; review correctness & docs Sub-task Resolved Steve Loughran
          18.
          distcp numListstatusThreads option doesn't get to -delete scan Sub-task Resolved Steve Loughran
          19.
          S3A Metrics in S3AInstrumentation Cause Memory Leaks in HBase Export Sub-task Resolved Unassigned
          20.
          S3A directory committer commit job fails if _temporary directory created under dest Sub-task Resolved Steve Loughran
          21.
          S3A staging committers to not log FNFEs on job abort listings Sub-task Resolved Unassigned
          22.
          S3AFileStatus to add a serialVersionUID; review & test serialization Sub-task Resolved Unassigned
          23.
          Expand S3A Assumed Role docs Sub-task Resolved Steve Loughran
          24.
          S3A Multipart Uploader to work with S3Guard and encryption Sub-task Resolved Ewan Higgs
          25.
          AssumedRoleCredentialProvider to propagate connection settings of S3A FS Sub-task Resolved Steve Loughran
          26.
          Stabilize S3A Assumed Role support Sub-task Resolved Steve Loughran
          27.
          Update aws-sdk version to 1.11.375 Sub-task Resolved Steve Loughran
          28.
          s3guard uploads command to add a way to complete outstanding uploads Sub-task Resolved Unassigned
          29.
          FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append. Sub-task Resolved Steve Loughran
          30.
          increase default value of fs.s3a.multipart.size to 128M Sub-task Resolved Unassigned
          31.
          Remove s3a user:secret authentication Sub-task Resolved Steve Loughran

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: