Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17566

Über-jira: S3A Hadoop 3.3.2 features

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Done
    • 3.3.1
    • 3.3.2
    • fs/s3
    • None

    Attachments

      Issue Links

        1.
        Add an Audit plugin point for S3A auditing/context Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 22h 20m
        2.
        S3A to treat "SdkClientException: Data read has a different length than the expected" as EOFException Sub-task Resolved Bogdan Stolojan

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1.5h
        3.
        Add a MkdirOperation for chained S3 operations during mkdir Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 50m
        4.
        Magic committer to downgrade abort in cleanup if list uploads fails with access denied Sub-task Resolved Bogdan Stolojan

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 0.5h
        5.
        S3Guard import can OOM on large imports Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 50m
        6.
        Support S3 Access Points Sub-task Resolved Bogdan Stolojan

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 16.5h
        7.
        Failure of ITestAssumeRole.testRestrictedCommitActions Sub-task Resolved Steve Loughran  
        8.
        Re-enable optimized copyFromLocal implementation in S3AFileSystem Sub-task Resolved Bogdan Stolojan

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 9h 50m
        9.
        S3A AWS Credential provider loading gets confused with isolated classloaders Sub-task Resolved Steve Loughran  
        10.
        S3A mkdirs to indicate which parent path element refers to a file Sub-task Resolved Unassigned  
        11.
        transient ITestS3AFileContextStatistics failure -read buffer not filled Sub-task Resolved Unassigned  
        12.
        s3a rename failed during copy, "Unable to copy part" + 200 error code Sub-task Resolved Unassigned  
        13.
        S3a: Failed to reset the request input stream/make S3A uploadPart() retriable Sub-task Resolved Unassigned  
        14.
        S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request Sub-task Resolved Unassigned  
        15.
        ITestS3AContractGetFileStatusV1List may have consistency issues Sub-task Resolved Unassigned  
        16.
        Distcp is unable to determine region with S3 PrivateLink endpoints Sub-task Resolved Unassigned  
        17.
        s3a to set fake directory marker contentType to application/x-directory Sub-task Resolved Steve Loughran  
        18.
        S3A copy/rename of large files to be parallelized as a multipart operation Sub-task Resolved Unassigned  
        19.
        S3A add histogram metrics types for latency, etc. Sub-task Resolved Sean Mackrory  
        20.
        s3guard to provide better diags on ddb init failures Sub-task Resolved Unassigned  
        21.
        s3a doesn't consider blobs with trailing / and content-length >0 as directories Sub-task Resolved Unassigned  
        22.
        Encrypt S3A data client-side with AWS SDK (S3-CSE) Sub-task Resolved Mehakmeet Singh

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 14h 40m
        23.
        declare that fs.s3a.ext. is a prefix for arbitrary extensions Sub-task Resolved Unassigned  
        24.
        improve setting of max connections in AWS client Sub-task Resolved Unassigned  
        25.
        AWS Data read stack trace in S3a putObjectDirect Sub-task Resolved Unassigned  
        26.
        increase performance of s3guard import command Sub-task Resolved Unassigned  
        27.
        S3A to add option fs.s3a.endpoint.region to set AWS region Sub-task Resolved Mehakmeet Singh

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h
        28.
        Upgrade aws-java-sdk to 1.11.1026 Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h
        29.
        S3A NetworkBinding has a runtime class dependency on a third-party shaded class Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 2h
        30.
        S3AFS creation fails "Unable to find a region via the region provider chain." Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h 50m
        31.
        bytesRead FS statistic showing twice the correct value in S3A Sub-task Resolved Mehakmeet Singh

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 2.5h
        32.
        S3A ITestPartialRenamesDeletes.testRenameDirFailsInDelete failure: missing directory marker Sub-task Resolved Steve Loughran  
        33.
        ITestS3AContractDistCp failing IllegalStateException: WFOPENSSL0030 Running handshake with buffered unwrapped data Sub-task Resolved Unassigned  
        34.
        fs.s3a.acl.default not working after S3A Audit feature added Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h
        35.
        S3A Tests to skip if S3Guard and S3-CSE are enabled. Sub-task Resolved Mehakmeet Singh

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h 40m
        36.
        S3A to support user-specified content encoding Sub-task Resolved Holden Karau

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 6h 20m
        37.
        S3A CSE: minor tuning Sub-task Resolved Mehakmeet Singh

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3.5h
        38.
        CredentialProviderFactory.getProviders() recursion loading JCEKS file from s3a Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 20m
        39.
        s3a: set fs.s3a.downgrade.syncable.exceptions = true by default Sub-task Resolved Steve Loughran

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3.5h
        40.
        AccessPoint verifyBucketExistsV2 always returns false Sub-task Resolved Bogdan Stolojan

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m

        Activity

          People

            stevel@apache.org Steve Loughran
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 103h 20m
                103h 20m