Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15220

Über-jira: S3a phase V: Hadoop 3.2 features

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0
    • None
    • None

    Description

      Über-jira for S3A work for Hadoop 3.2.x

      The items from HADOOP-14831 which didn't get into Hadoop-3.1, and anything else

      Attachments

        Issue Links

        1.
        add htrace context to HTTP requests as ? parameter Sub-task Resolved Unassigned Actions
        2.
        S3ADataBlocks.DiskBlock to lazy create dest file for faster 0-byte puts Sub-task Resolved Unassigned Actions
        3.
        use s3 tags/headers to record permissions on objects, so preserving them through distcp round trips Sub-task Resolved Unassigned Actions
        4.
        Automatically determine region & hence S3 endpoint of buckets Sub-task Resolved Unassigned Actions
        5.
        S3A shell entry point to support commands specific to S3A. Sub-task Resolved Unassigned Actions
        6.
        Hive doesn't let s3a patch the credential provider path Sub-task Resolved Unassigned Actions
        7.
        s3a encryption tests fail when dest bucket has SSE-KMS enabled Sub-task Resolved Unassigned Actions
        8.
        AWSCredentialProviderList to throw custom NoCredentialsException; retry logic to handle Sub-task Resolved Unassigned Actions
        9.
        Add Private/Unstable BulkDelete operations to supporting object stores for DistCP Sub-task Resolved Steve Loughran Actions
        10.
        S3AInputStream: Handle http stream skip(n) skipping < n bytes in a forward seek Sub-task Resolved Unassigned Actions
        11.
        S3ABlockOutputStream.flush() be no-op when stream closed Sub-task Resolved Gabor Bota Actions
        12.
        ITestS3AEmptyDirectory.testParallelJobsToAdjacentPaths failure, FileAlreadyExistsException Sub-task Resolved Steve Loughran Actions
        13.
        regression: secret stripping from S3x URIs breaks some downstream code Sub-task Resolved Vinayakumar B Actions
        14.
        S3A ITestS3GuardWriteBack failing if bucket explicitly set to s3guard+DDB Sub-task Resolved Steve Loughran Actions
        15.
        S3 returning 400 on the directory /test/ GET of getFileStatus Sub-task Resolved Steve Loughran Actions
        16.
        S3ARetryPolicy for 400/BadArgument to be "fail" Sub-task Resolved Steve Loughran Actions
        17.
        Stabilize/tune S3A committers; review correctness & docs Sub-task Resolved Steve Loughran Actions
        18.
        distcp numListstatusThreads option doesn't get to -delete scan Sub-task Resolved Steve Loughran Actions
        19.
        S3A Metrics in S3AInstrumentation Cause Memory Leaks in HBase Export Sub-task Resolved Unassigned Actions
        20.
        S3A directory committer commit job fails if _temporary directory created under dest Sub-task Resolved Steve Loughran Actions
        21.
        S3A staging committers to not log FNFEs on job abort listings Sub-task Resolved Unassigned Actions
        22.
        S3AFileStatus to add a serialVersionUID; review & test serialization Sub-task Resolved Unassigned Actions
        23.
        Expand S3A Assumed Role docs Sub-task Resolved Steve Loughran Actions
        24.
        S3A Multipart Uploader to work with S3Guard and encryption Sub-task Resolved Ewan Higgs Actions
        25.
        AssumedRoleCredentialProvider to propagate connection settings of S3A FS Sub-task Resolved Steve Loughran Actions
        26.
        Stabilize S3A Assumed Role support Sub-task Resolved Steve Loughran Actions
        27.
        Update aws-sdk version to 1.11.375 Sub-task Resolved Steve Loughran Actions
        28.
        s3guard uploads command to add a way to complete outstanding uploads Sub-task Resolved Unassigned Actions
        29.
        FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append. Sub-task Resolved Steve Loughran Actions
        30.
        increase default value of fs.s3a.multipart.size to 128M Sub-task Resolved Unassigned Actions
        31.
        Remove s3a user:secret authentication Sub-task Resolved Steve Loughran Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stevel@apache.org Steve Loughran
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment