Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16829

Über-jira: S3A Hadoop 3.4 features

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:

      Description

      Über-jira: S3A features/fixes for Hadoop 3.4

      As usual, this will clutter up with everything which hasn't gone in: don't interpret presence on this list as a commitment to implement.

      And for anyone wanting to add patches

      MUST

      1. reviews via github PRs
      2. no declaration of AWS S3 endpoint (or other S3 impl) -no review

      SHOULD

      1. have a setup for testing SSE-KMS, DDB/S3Guard
      2. including an assumed role we can use for AssumedRole Delegation Tokens

      If you are going near those bits of code, they uprate from SHOULD to MUST.

        Attachments

          Issue Links

          1.
          S3AInputStream logging to make it easier to debug file leakage Sub-task Open Unassigned  
          2.
          Filesystem discovery to stop loading implementation classes Sub-task Open Unassigned  
          3.
          s3a rename failed during copy, "Unable to copy part" + 200 error code Sub-task Open Unassigned  
          4.
          make sure staging committers collect DTs for the staging FS Sub-task Open Unassigned  
          5.
          Tune S3AFileSystem.listLocatedStatus Sub-task Resolved Mukund Thakur  
          6.
          Remove transient dependency on hadoop-hdfs-client Sub-task Open Unassigned  
          7.
          s3a directory housekeeping operations to be done in async thread Sub-task Open Unassigned  
          8.
          s3a test can hang in teardown with network problems Sub-task Resolved Unassigned  
          9.
          distcp -update to S3A; abfs, etc always overwrites due to block size mismatch Sub-task Resolved Steve Loughran  
          10.
          S3A input stream to support ByteBufferReadable Sub-task Open Unassigned  
          11.
          Encrypt S3A buffered data on disk Sub-task Open Unassigned  
          12.
          S3a: Failed to reset the request input stream/make S3A uploadPart() retriable Sub-task Open Unassigned  
          13.
          S3AFileSystem.getContentSummary() to use listFiles(recursive) Sub-task Open Unassigned  
          14.
          add extra S3A MPU test to see what happens if a file is created during the MPU Sub-task Resolved Steve Loughran  
          15.
          support git-secrets commit hook to keep AWS secrets out of git Sub-task Patch Available Steve Loughran  
          16.
          Tune hadoop-aws parallel test surefire/failsafe settings Sub-task Open Unassigned  
          17.
          S3A Secret access to fall back to XML if credential provider raises IOE. Sub-task Open Unassigned  
          18.
          S3A doesn't actually verify paths have the correct authority Sub-task Resolved Steve Loughran  
          19.
          Use error code detail in AWS server responses for finer grained exceptions Sub-task Open Unassigned  
          20.
          Test MR split optimisation with recursive listing Sub-task Open Unassigned  
          21.
          initial part uploads seem to block unnecessarily in S3ABlockOutputStream Sub-task Open Steven Rand  
          22.
          Speed up S3A test runs Sub-task Open Unassigned  
          23.
          S3AFilesystem.initiateRename() can skip check on dest.parent status if src has same parent Sub-task Resolved Steve Loughran  
          24.
          S3A copy/rename of large files to be parallelized as a multipart operation Sub-task Open Unassigned  
          25.
          Test hadoop fs shell against s3a; fix problems Sub-task Open Unassigned  
          26.
          s3guard bucket-info command to add a verify-property <key>=<value> <bucket> Sub-task Open Unassigned  
          27.
          S3A to implement rename(final Path src, final Path dst, final Rename... options) Sub-task Open Unassigned  
          28.
          hook up AwsSdkMetrics to hadoop metrics Sub-task In Progress Steve Loughran  
          29.
          Add some tests about S3 timestamp tracking Sub-task Open Unassigned  
          30.
          S3a operations keep retrying if the password is wrong Sub-task Open Thomas Poepping  
          31.
          Add some S3A-specific create file options Sub-task Open Unassigned  
          32.
          s3a to set fake directory marker contentType to application/x-directory Sub-task Open Steve Loughran  
          33.
          Report problems w/ local S3A buffer directory meaningfully Sub-task Open Unassigned  
          34.
          S3A Filesystem does not check return from AmazonS3Client deleteObjects Sub-task Open Unassigned  
          35.
          Optimize uses of FS operations in the ASF analysis frameworks and libraries Sub-task Open Steve Loughran  
          36.
          Possible NPE in S3A MultiObjectDeleteSupport error handling Sub-task Open Steve Loughran  
          37.
          shell rm command to not rename to ~/.Trash in object stores Sub-task Open Unassigned  
          38.
          Add S3A support for Async Scatter/Gather IO Sub-task Open Gabor Bota  
          39.
          Impersonate hosts in s3a for better data locality handling Sub-task Open Thomas Demoor  
          40.
          log DNS addresses on s3a init Sub-task Resolved Mukund Thakur  
          41.
          export s3a BlockingThreadPoolExecutorService pool info (size, load) as gauges Sub-task Open Unassigned  
          42.
          increase the default number of threads and http connections in S3A Sub-task Open Unassigned  
          43.
          S3A authenticators to log origin of .secret.key options Sub-task Resolved Unassigned  
          44.
          Support AWS S3 reduced redundancy storage class Sub-task Open Unassigned  
          45.
          s3a doesn't consider blobs with trailing / and content-length >0 as directories Sub-task Open Unassigned  
          46.
          make s3a read fault injection configurable including "off" Sub-task Open Unassigned  
          47.
          Retrive modtime of PUT file from store, via response or HEAD Sub-task Resolved Unassigned  
          48.
          s3a create() doesn't check for an ancestor path being a file Sub-task Resolved Sean Mackrory  
          49.
          strip s3.amazonaws.com off hostnames before making s3a calls Sub-task Open Unassigned  
          50.
          add a special 0 byte input stream for empty blobs Sub-task Open Unassigned  
          51.
          improve setting of max connections in AWS client Sub-task Open Unassigned  
          52.
          multipart/huge file upload tests to look at checksums returned Sub-task Open Unassigned  
          53.
          Some S3A tests leak filesystem instances Sub-task Open Unassigned  
          54.
          s3guard to provide better diags on ddb init failures Sub-task Open Unassigned  
          55.
          Improve isolation of FS instances in S3A committer tests Sub-task Open Unassigned  
          56.
          s3a rm on the CLI generates deprecation warning on io.bytes.per.checksum Sub-task Open Unassigned  
          57.
          S3A getContentSummary() to move to listFiles(recursive) to count children; instrument use Sub-task Open Unassigned  
          58.
          review S3A translateException translation matches IBM CORS spec Sub-task Open Unassigned  
          59.
          Add fs.s3a.rename.raises.exceptions to raise exceptions on rename failures Sub-task Open Unassigned  
          60.
          add tests/docs for HAR files on s3a Sub-task Open Unassigned  
          61.
          S3A to use a thread pool for async path operations Sub-task Open Unassigned  
          62.
          builld up md5 checksum as blocks are built in S3ABlockOutputStream; validate upload Sub-task Open Unassigned  
          63.
          FileSystem/s3a processDeleteOnExit to skip the exists() check Sub-task Open Unassigned  
          64.
          Add custom InstanceProfileCredentialsProvider with more resilience to throttling Sub-task Open Unassigned  
          65.
          ITestS3AMiniYarnCluster fails on sequential runs with Kerberos error Sub-task Open Unassigned  
          66.
          S3a DelegationToken bindings to to support a "correlation ID" for the UA header Sub-task Open Unassigned  
          67.
          add experimental optimization of s3a directory marker handling Sub-task Resolved Unassigned  
          68.
          Report on S3A cached 404 recovery better Sub-task Open Unassigned  
          69.
          Stabilize openFile() and adopt internally Sub-task In Progress Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 10m
          70.
          S3AInputStream.skip() to use lazy seek Sub-task Open Unassigned  
          71.
          s3a to instrument duration of HTTP calls Sub-task Open Unassigned  
          72.
          Document `dynamodb:TagResource` an explicit client-side permission for S3Guard Sub-task Open Gabor Bota  
          73.
          catch and downgrade all exceptions trying to load openssl native libs through wildfly Sub-task Resolved Steve Loughran  
          74.
          ITestS3ARemoteFileChanged doesn't overwrite test data creation Sub-task Open Unassigned  
          75.
          Optimize getFileStatus in S3A Sub-task Resolved Steven K. Wong  
          76.
          s3a mkdir path/ can add 404 to S3 load balancers Sub-task Open Unassigned  
          77.
          AWS Data read stack trace in S3a putObjectDirect Sub-task Open Unassigned  
          78.
          AmazonClient 30x exceptions to include redirect URL in message Sub-task Open Unassigned  
          79.
          S3A FS to add "s3a:no-existence-checks" to the builder file creation option set Sub-task Open Unassigned  
          80.
          s3a new getdefaultblocksize be called in getFileStatus which has not been implemented in s3afilesystem yet Sub-task Open Unassigned  
          81.
          Use lighter-weight alternatives to innerGetFileStatus where possible Sub-task Open Unassigned  
          82.
          S3A Input Stream bytes read counter isn't getting through to StorageStatistics/instrumentation properly Sub-task Open Unassigned  
          83.
          job commit failure in S3A MR magic committer test Sub-task Resolved Steve Loughran  
          84.
          intermittent failure of ITestS3GuardListConsistency.testInconsistentS3ClientDeletes in parallel runs Sub-task Open Unassigned  
          85.
          ITestDynamoDBMetadataStore.testTableVersioning failure -DDB deleteItem consistency? Sub-task Open Unassigned  
          86.
          Add S3AWriteOpContext for write ops; pass in statistics and other settings Sub-task Open Unassigned  
          87.
          S3A: Set thread names with more specific information about the call. Sub-task Open Unassigned  
          88.
          S3A Client to add explicit support for versioned stores Sub-task Resolved Steve Loughran  
          89.
          Add AWS S3 Transfer acceleration support Sub-task Open Unassigned  
          90.
          Add a way for an FS instance to say "really, no trash interval at all" Sub-task Open Unassigned  
          91.
          s3guard uploads command to list date and initiator of outstanding uploads Sub-task Open Unassigned  
          92.
          test and document use of fs.s3a.signing-algorithm Sub-task Open Unassigned  
          93.
          S3A statistic collection underrecords bytes written in helper threads Sub-task In Progress Steve Loughran  
          94.
          S3AInputStream.seek should throw EOFException if seeking past the end of file Sub-task Open Unassigned  
          95.
          S3a auth exception to link to a wiki page on the problem Sub-task Open Unassigned  
          96.
          s3a to improve diags on s3a bad request message Sub-task Open Unassigned  
          97.
          AbstractContractDistCpTest to test attr preservation with -p, verify blobstores downgrade Sub-task Open Steve Loughran  
          98.
          s3a: auto-detect region for bucket and use right endpoint Sub-task Open Aaron Fabbri  
          99.
          ITestS3AContractRootDir failure on non-S3Guarded bucket Sub-task Open Unassigned  
          100.
          s3guard bucket-info command to include default bucket encryption info Sub-task Open Unassigned  
          101.
          S3 SSEC tests to downgrade when running against a mandatory encryption object store Sub-task Open Unassigned  
          102.
          S3AInputStream read(bytes[]) to not retry on read failure: pass action up Sub-task Open Unassigned  
          103.
          typo in TestNeworkBinding Sub-task Open Steve Loughran  
          104.
          ITestS3A select tests fail if user kinited in Sub-task Open Unassigned  
          105.
          S3A add histogram metrics types for latency, etc. Sub-task Open Sean Mackrory  
          106.
          cherry pick s3 ehancements from PrestoS3FileSystem Sub-task Open Unassigned  
          107.
          S3aDelegationTokens to add accessor for tests to get at the token binding Sub-task Open Unassigned  
          108.
          s3guard LimitExceededException -too many tables Sub-task Open Unassigned  
          109.
          Support multipart download in S3AFileSystem Sub-task Open Unassigned  
          110.
          NPE in S3AInputStream.read() in ITestS3AInconsistency.testOpenFailOnRead Sub-task Open Unassigned  
          111.
          S3A DT marshalling to include nested error text in wrapped message Sub-task Open Unassigned  
          112.
          S3 Select Exceptions are not being converted to IOEs Sub-task Open Unassigned  
          113.
          remove misleading fs.s3a.delegation.tokens.enabled prompt Sub-task Open Unassigned  
          114.
          S3A to support configuring various AWS S3 client extended options Sub-task Open Unassigned  
          115.
          ITestS3ARemoteFileChanged tests fail if you set the bucket to versionid tracking Sub-task Resolved Unassigned  
          116.
          Review S3A documentation to make sure it is consistent with the current codebase Sub-task Open Unassigned  
          117.
          S3A DT support to warn when loading expired token Sub-task Open Steve Loughran  
          118.
          ITestS3AAWSCredentialsProvider tests fail if a bucket has DTs enabled Sub-task Open Unassigned  
          119.
          Clarify committers.md around v2 failure handling Sub-task Open Unassigned  
          120.
          S3AFileStatus to add a serialVersionUID; review & test serialization Sub-task Open Unassigned  
          121.
          test YARN log collection works to s3a Sub-task Open Unassigned  
          122.
          Encrypt S3A data client-side with AWS SDK (S3-CSE) Sub-task Patch Available Igor Mazur  
          123.
          S3A can support short user-friendly aliases for configuration of credential providers. Sub-task Open Unassigned  
          124.
          Add some Java-8 friendly way to work with RemoteIterable, especially listings Sub-task Open Unassigned  
          125.
          Handle S3A "glacier" data Sub-task Open Unassigned  
          126.
          Add common getFileBlockLocations() emulation for object stores, including S3A Sub-task Patch Available Steve Loughran  
          127.
          S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes Sub-task Patch Available Kazuyuki Tanimura  
          128.
          s3a mkdirs() to not check dest for a dir marker Sub-task Open Unassigned  
          129.
          S3A to support Requester Pays Buckets Sub-task Patch Available Mandus Momberg

          0%

          Original Estimate - 2h
          Remaining Estimate - 2h
          130.
          clean up ITestS3AFileSystemContract Sub-task Patch Available Unassigned  
          131.
          ITestS3AContractGetFileStatusV1List may have consistency issues Sub-task Open Unassigned  
          132.
          S3ARetryPolicy to handle AWS 500 responses/error code TooBusyException with the throttle backoff policy Sub-task Open Unassigned  
          133.
          S3A FS deleteOnExit to skip the exists check Sub-task Resolved Unassigned  
          134.
          S3A to optionally retain directory markers Sub-task Resolved Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          135.
          Add some Abortable.abort() interface for streams etc which can be terminated Sub-task Open Unassigned  
          136.
          S3A mkdirs to indicate which parent path element refers to a file Sub-task Open Unassigned  
          137.
          NPE in s3a byte buffer block upload Sub-task Open Unassigned  
          138.
          declare that fs.s3a.ext. is a prefix for arbitrary extensions Sub-task Open Unassigned  
          139.
          AWS AssumedRoleCredentialProvider needs ExternalId add Sub-task Open Unassigned  
          140.
          IAM role created by S3A DT doesn't include DynamoDB scan Sub-task Open Unassigned  
          141.
          S3A FullCredentialsTokenBinding fails if local credentials are unset Sub-task Resolved Steve Loughran  
          142.
          S3AFilesystem trash handling should respect the current UGI Sub-task Open Unassigned  
          143.
          S3AInputStream.remainingInFile should use nextReadPos Sub-task Reopened lqjacklee  
          144.
          s3guard can't init table if caller doesn't have tag permissions Sub-task Open Unassigned  
          145.
          ITestS3AContractSeek teardown closes test FS before superclass can do its cleanup Sub-task Open Unassigned  
          146.
          Possible inconsistent state of AbstractDelegationTokenSecretManager Sub-task Patch Available Hankó Gergely  
          147.
          ITestCustomSigner uses absolute paths off the bucket root rather than fork-relative Sub-task Open Unassigned  
          148.
          ITestS3GuardOutOfBandOperations.testListingDelete failing on versioned bucket Sub-task Resolved Steve Loughran  
          149.
          log accepted/rejected fs.s3a.authoritative.path paths @ debug Sub-task Open Unassigned  
          150.
          Add option for a prefix to put in front of every s3guard table Sub-task Open Unassigned  
          151.
          Add more s3guard metrics Sub-task Open Unassigned  
          152.
          Improve DynamoDB schema update story Sub-task Open Sean Mackrory  
          153.
          S3Guard: Optimize performance of handling OOB operations in non-authoritative mode Sub-task Open Unassigned  
          154.
          getFileChecksum() needs to adopt S3Guard Sub-task In Progress lqjacklee  
          155.
          reduce/tune read failure fault injection on inconsistent client Sub-task Open Unassigned  
          156.
          increase performance of s3guard import command Sub-task Open Unassigned  
          157.
          intermittent failure of ITestCommitOperations: too many s3guard writes Sub-task Open Unassigned  
          158.
          S3a getFileStatus to update DDB if an S3 query returns etag/versionID Sub-task Open Unassigned  
          159.
          Possible for modified configuration to leak into metadatastore in S3GuardTool Sub-task Open Unassigned  
          160.
          S3Guard instrumentation to include cost of DynamoDB ops as metric Sub-task Open Unassigned  
          161.
          S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request Sub-task Open Unassigned  
          162.
          Intermittent failure of ITestS3GuardConcurrentOps#testConcurrentTableCreations Sub-task Open Unassigned  
          163.
          Intermittent failure of ITestS3GuardToolDynamoDB#testDynamoDBInitDestroyCycle Sub-task Open Unassigned  
          164.
          S3Guard init command uses global settings, not those of target bucket Sub-task Reopened Steve Loughran  
          165.
          Improve throttling on S3Guard DDB batch retries Sub-task Open Unassigned  
          166.
          S3guard: add inconsistency detection metrics Sub-task Open Unassigned  
          167.
          S3Guard prune to only remove auth dir marker if files (not tombstones) are removed Sub-task Open Unassigned  
          168.
          ZookeeperMetadataStore: Use Zookeeper as S3Guard backend store Sub-task Open Unassigned  
          169.
          tag S3GuardTool entry points as limitedPrivate("management-tools")/evolving Sub-task Open Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          170.
          Fix ITestS3GuardToolLocal#testInitNegativeRead test failure Sub-task Open Steve Loughran  
          171.
          Ensure controls in-place to prevent clients with significant clock skews pruning aggressively Sub-task Open Unassigned  
          172.
          Scheme assertion in S3Guard DynamoDBMetadataStore::checkPath is unnecessarily restrictive Sub-task Open Unassigned  
          173.
          improvements to S3GuardTool destroy command Sub-task Open Unassigned  
          174.
          Clock skew can cause S3Guard to think object metadata is out of date Sub-task Open Unassigned  
          175.
          S3guard metadata stores to support millions of entries Sub-task Open Unassigned  
          176.
          ITestS3GuardToolDynamoDB.testDynamoDBInitDestroyCycle fails if test bucket isn't on demand Sub-task Open Steve Loughran  
          177.
          S3Guard to self update on directory listings of S3 Sub-task Open Unassigned  
          178.
          S3guard mistakes root URI without / as non-absolute path Sub-task Open Unassigned  
          179.
          HADOOP-16953. tune s3guard disabled warnings Sub-task Resolved Steve Loughran  
          180.
          mkdir on s3a should not be sensitive to trailing '/' Sub-task Open Unassigned  
          181.
          s3a to not need wildfly on the classpath Sub-task Resolved Steve Loughran  
          182.
          ITestS3AConfiguration proxy tests fail when bucket probes == 0 Sub-task Open Mukund Thakur  
          183.
          Tune S3A listFiles() api. Sub-task Resolved Mukund Thakur  
          184.
          Tune listStatus() api of s3a. Sub-task Resolved Mukund Thakur

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 40m
          185.
          S3A to support additional token issuers Sub-task Resolved Steve Loughran  
          186.
          S3A deleteObjects hanging/retrying forever Sub-task Open Unassigned  
          187.
          S3A staging committer committing duplicate files Sub-task Open Steve Loughran  
          188.
          S3A delegation token binding to support secondary binding list Sub-task In Progress Steve Loughran  
          189.
          Optimise s3a Listing to be fully asynchronous. Sub-task Resolved Mukund Thakur  
          190.
          whitespace not allowed in paths when saving files to s3a via committer Sub-task Open Unassigned  
          191.
          Refactor S3A Listing code for better isolation Sub-task Resolved Mukund Thakur  
          192.
          S3AFileSystem.listLocatedStatu(file) does a LIST even with S3Guard Sub-task Open Mukund Thakur  
          193.
          ITestS3ADirectoryPerformance.testListOperations failing Sub-task Resolved Mukund Thakur  
          194.
          Re-enable optimized copyFromLocal implementation in S3AFileSystem Sub-task Open Unassigned  
          195.
          ITestS3GuardOutOfBandOperations testListingDelete[auth=false] fails on unversioned bucket Sub-task Open Unassigned  
          196.
          S3A client retries on SSL Auth exceptions triggered by "." bucket names Sub-task Open Unassigned  
          197.
          S3AFileSystem silently deletes "fake" directories when writing a file. Sub-task Resolved Unassigned  
          198.
          ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS Sub-task Resolved Mukund Thakur

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m
          199.
          Handle transient stream read failures in FileSystem contract tests Sub-task Resolved Unassigned

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 10m
          200.
          add way for s3a to recognise buckets with "." in name and switch to path access Sub-task Resolved Unassigned  
          201.
          Decrease size of s3a dependencies Sub-task Resolved Unassigned  
          202.
          Support S3 Access Points Sub-task Open Unassigned  
          203.
          Backport HADOOP-13230 list/getFileStatus changes for preserved directory markers Sub-task Resolved Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 50m
          204.
          Renaming a file under a sibling empty directory doesn't delete dest dir's marker Sub-task Resolved Steve Loughran  
          205.
          ITestS3AConfiguration.testProxyConnection failing when s3a bucket probe disabled Sub-task Open Unassigned  
          206.
          Failure of ITestAssumeRole.testRestrictedCommitActions Sub-task Open Steve Loughran  
          207.
          improve s3guard markers command line tool Sub-task Resolved Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 10m
          208.
          Backport HADOOP-13230 listing changes for preserved directory markers to 3.1.x Sub-task Resolved Steve Loughran  
          209.
          S3A (async) ObjectListingIterator to block in hasNext() for results Sub-task Open Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1.5h
          210.
          HADOOP-17244. S3A directory delete tombstones dir markers prematurely. Sub-task Reopened Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4.5h
          211.
          Distcp to set S3 Storage Class Sub-task Open Unassigned

          0%

          Original Estimate - 168h
          Remaining Estimate - 168h
          212.
          s3a rename() now requires s3:deleteObjectVersion permission Sub-task Resolved Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 20m
          213.
          transient ITestS3AFileContextStatistics failure -read buffer not filled Sub-task Open Unassigned  
          214.
          S3A to always probe S3 in S3A getFileStatus on non-auth paths Sub-task Resolved Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3h 40m
          215.
          ITestCustomSigner fails with gcs s3 compatible endpoint. Sub-task Open Mukund Thakur

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          216.
          S3AInputStream to be resilient to faiures in abort(); translate AWS Exceptions Sub-task Resolved Unassigned  
          217.
          FileSystem.get to support slow-to-instantiate FS clients Sub-task In Progress Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 40m
          218.
          S3A committer to support concurrent jobs with same app attempt ID & dest dir Sub-task In Progress Steve Loughran

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1.5h
          219.
          s3a getFileStatus("/") to skip IO Sub-task Open Unassigned  
          220.
          S3A marker tool mixes up -min and -max Sub-task In Progress Steve Loughran  
          221.
          s3a listing operation will fail in async prefetch if fs closed Sub-task Open Mukund Thakur  

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 170h Original Estimate - 170h
                  170h
                  Remaining:
                  Time Spent - 30h 50m Remaining Estimate - 170h
                  170h
                  Logged:
                  Time Spent - 30h 50m Remaining Estimate - 170h
                  30h 50m