Description
HADOOP-11571 covered the core s3a bugs surfacing in Hadoop-2.6 & other enhancements to improve S3 (performance, proxy, custom endpoints)
This JIRA covers post-2.7 issues and enhancements.
Attachments
Issue Links
- depends upon
-
HADOOP-12269 Update aws-sdk dependency to 1.10.6; move to aws-sdk-s3
- Resolved
-
HADOOP-13826 S3A Deadlock in multipart copy due to thread pool limits.
- Resolved
-
HADOOP-14028 S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures
- Resolved
-
HADOOP-12346 Increase some default timeouts / retries for S3a connector
- Resolved
-
HADOOP-12667 s3a: Support createNonRecursive API
- Resolved
-
HADOOP-12851 S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders
- Resolved
-
HADOOP-12970 Intermittent signature match failures in S3AFileSystem due connection closure
- Resolved
-
HADOOP-12979 IOE in S3a: ${hadoop.tmp.dir}/s3a not configured
- Resolved
-
HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials
- Resolved
-
HADOOP-3733 "s3:" URLs break when Secret Key contains a slash, even if encoded
- Resolved
-
HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories
- Resolved
-
HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs()
- Resolved
-
HADOOP-13516 Listing an empty s3a NON root directory throws FileNotFound.
- Resolved
-
HADOOP-13993 transient failure of ITestS3AInputStreamPerformance.testDecompressionSequential128K
- Resolved
-
HADOOP-11571 Über-jira: S3a stabilisation phase I
- Closed
-
HADOOP-12810 FileSystem#listLocatedStatus causes unnecessary RPC calls
- Closed
-
HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider
- Resolved
-
HADOOP-12537 S3A to support Amazon STS temporary credentials
- Resolved
-
HADOOP-11687 Ignore x-* and response headers when copying an Amazon S3 object
- Resolved
-
HADOOP-11827 Speed-up distcp buildListing() using threadpool
- Resolved
-
HADOOP-12891 S3AFileSystem should configure Multipart Copy threshold and chunk size
- Resolved
-
HADOOP-13017 Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0
- Resolved
-
HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata.
- Resolved
-
HADOOP-13239 Deprecate s3:// in branch-2
- Resolved
-
HADOOP-12807 S3AFileSystem should read AWS credentials from environment variables
- Resolved
-
HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint
- Resolved
-
HADOOP-12982 Document missing S3A and S3 properties
- Resolved
-
HADOOP-13166 add getFileStatus("/") test to AbstractContractGetFileStatusTest
- Resolved
-
HADOOP-13169 Randomize file list in SimpleCopyListing
- Resolved
-
HADOOP-13323 Downgrade stack trace on FS load from Warn to debug
- Resolved
-
HADOOP-13139 Branch-2: S3a to use thread pool that blocks clients
- Resolved
- is blocked by
-
HADOOP-14204 S3A multipart commit failing, "UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort"
- Resolved
- is depended upon by
-
HIVE-14269 Performance optimizations for data on S3
- Open
-
HADOOP-13204 Über-jira: S3a phase III: scale and tuning
- Resolved
- relates to
-
HADOOP-13113 Enable parallel test execution for hadoop-aws.
- Resolved
-
HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem
- Resolved
-
HADOOP-12548 Read s3a creds from a Credential Provider
- Resolved