Bulk Operation

  1. Choose Issues
  2. Choose Operation
  3. Operation Details
  4. Confirmation

Step 1 of 4: Choose Issues

Cancel

T Patch Info Key Summary Assignee Reporter P Status Resolution Created Updated Due Development
Sub-task HADOOP-17401

HADOOP-11694 GCS to support per-bucket configuration

Steve Loughran t oo Major Resolved Duplicate  
Sub-task HADOOP-14081

HADOOP-11694 S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock)

Rajesh Balamohan Rajesh Balamohan Minor Resolved Fixed  
Sub-task HADOOP-14019

HADOOP-11694 fix some typos in the s3a docs

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13871

HADOOP-11694 ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13857

HADOOP-11694 S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13855

HADOOP-11694 Fix a couple of the s3a statistic names to be consistent with the rest

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13823

HADOOP-11694 s3a rename: fail if dest file exists

Steve Loughran Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-13801

HADOOP-11694 regression: ITestS3AMiniYarnCluster failing

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13772

HADOOP-11694 ITestS3AContractRootDir still playing up, bug in eventually() retry logic?

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13755

HADOOP-11694 Purge superfluous/obsolete S3A Tests

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13735

HADOOP-11694 ITestS3AFileContextStatistics.testStatistics() failing

Pieter Reuse Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13729

HADOOP-11694 switch to Configuration.getLongBytes for byte options

Abhishek Modi Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13727

HADOOP-11694 S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider.

Chris Nauroth Rajesh Balamohan Minor Resolved Fixed  
Sub-task HADOOP-13703

HADOOP-11694 S3ABlockOutputStream to pass Yetus & Jenkins

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13692

HADOOP-11694 hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts.

Chris Nauroth Chris Nauroth Minor Resolved Fixed  
Sub-task HADOOP-13680

HADOOP-11694 fs.s3a.readahead.range to use getLongBytes

Abhishek Modi Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13674

HADOOP-11694 S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint.

Chris Nauroth Chris Nauroth Minor Resolved Fixed  
Sub-task HADOOP-13655

HADOOP-11694 document object store use with fs shell and distcp

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13621

HADOOP-11694 s3:// should have been fully cut off from trunk

Mingliang Liu Mingliang Liu Major Resolved Fixed  
Sub-task HADOOP-13614

HADOOP-11694 Purge some superfluous/obsolete S3 FS tests that are slowing test runs down

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13599

HADOOP-11694 s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13569

HADOOP-11694 S3AFastOutputStream to take ProgressListener in file create()

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13567

HADOOP-11694 S3AFileSystem to override getStorageStatistics() and so serve up its statistics

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13566

HADOOP-11694 NPE in S3AFastOutputStream.write

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13560

HADOOP-11694 S3ABlockOutputStream to support huge (many GB) file writes

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13541

HADOOP-11694 explicitly declare the Joda time version S3A depends on

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13540

HADOOP-11694 improve section on troubleshooting s3a auth problems

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13531

HADOOP-11694 S3A output streams to share a single LocalDirAllocator for round-robin drive use

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13405

HADOOP-11694 doc for “fs.s3a.acl.default” indicates incorrect values

Shen Yinjie Shen Yinjie Minor Resolved Fixed  
Sub-task HADOOP-13336

HADOOP-11694 S3A to support per-bucket configuration

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13324

HADOOP-11694 s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints)

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13319

HADOOP-11694 S3A to list InstanceProfileCredentialsProvider after EnvironmentVariableCredentialsProvider

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13313

HADOOP-11694 S3A TemporaryAWSCredentialsProvider to support Hadoop Credential providers for secrets

Unassigned Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13310

HADOOP-11694 S3A reporting of file group as empty is harmful to compatibility for the shell.

Unassigned Chris Nauroth Minor Resolved Duplicate  
Sub-task HADOOP-13309

HADOOP-11694 Document S3A known limitations in file ownership and permission model.

Chris Nauroth Chris Nauroth Minor Resolved Fixed  
Sub-task HADOOP-13286

HADOOP-11694 add a S3A scale test to do gunzip and linecount

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13281

HADOOP-11694 S3A to track multipart upload count, size, duration

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13275

HADOOP-11694 hadoop fs command path doesn't include translation of amazon client exceptions

Steve Loughran Steve Loughran Minor Resolved Invalid  
Sub-task HADOOP-13262

HADOOP-11694 set multipart delete timeout to 5 * 60s in S3ATestUtils.createTestFileSystem

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13259

HADOOP-11694 mvn fs/s3 test runs to set DNS TTL to 20s

Unassigned Steve Loughran Minor Resolved Won't Fix  
Sub-task HADOOP-13252

HADOOP-11694 Tune S3A provider plugin mechanism

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13241

HADOOP-11694 document s3a better

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13237

HADOOP-11694 s3a initialization against public bucket fails if caller lacks any credentials

Chris Nauroth Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13231

HADOOP-11694 Isolate test path used by a few S3A tests for more reliable parallel execution.

Steve Loughran Chris Nauroth Minor Resolved Duplicate  
Sub-task HADOOP-13212

HADOOP-11694 Provide an option to set the socket buffers in S3AFileSystem

Rajesh Balamohan Rajesh Balamohan Minor Resolved Fixed  
Sub-task HADOOP-13208

HADOOP-11694 S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13203

HADOOP-11694 S3A: Support fadvise "random" mode for high performance readPositioned() reads

Rajesh Balamohan Rajesh Balamohan Major Resolved Fixed  
Sub-task HADOOP-13188

HADOOP-11694 S3A file-create should throw error rather than overwrite directories

Steve Loughran Raymie Stata Minor Resolved Fixed  
Sub-task HADOOP-13171

HADOOP-11694 Add StorageStatistics to S3A; instrument some more operations

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13162

HADOOP-11694 Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs

Rajesh Balamohan Rajesh Balamohan Minor Resolved Fixed  
Sub-task HADOOP-13158

HADOOP-11694 S3AFileSystem#toString might throw NullPointerException due to null cannedACL.

Chris Nauroth Chris Nauroth Minor Resolved Fixed  
Sub-task HADOOP-13154

HADOOP-11694 S3AFileSystem printAmazonServiceException/printAmazonClientException appear copy & paste of AWS examples

Steve Loughran Steve Loughran Blocker Closed Fixed  
Sub-task HADOOP-13131

HADOOP-11694 Add tests to verify that S3A supports SSE-S3 encryption

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13130

HADOOP-11694 s3a failures can surface as RTEs, not IOEs

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13122

HADOOP-11694 Customize User-Agent header sent in HTTP requests by S3A.

Chris Nauroth Chris Nauroth Minor Resolved Fixed  
Sub-task HADOOP-13113

HADOOP-11694 Enable parallel test execution for hadoop-aws.

Chris Nauroth Chris Nauroth Minor Resolved Fixed  
Sub-task HADOOP-13106

HADOOP-11694 if fs.s3a.block.size option == 0, use partition size option for blocksize

Steve Loughran Steve Loughran Minor Resolved Won't Fix  
Sub-task HADOOP-13078

HADOOP-11694 switch hadoop-aws back to using the (heavy) amazon-sdk JAR

Steve Loughran Steve Loughran Major Resolved Won't Fix  
Sub-task HADOOP-13062

HADOOP-11694 S3A Introspect to invoke incompatible AWS TransferManagerConfiguration methods

Unassigned Steve Loughran Major Resolved Won't Fix  
Sub-task HADOOP-13058

HADOOP-11694 S3A FS fails during init against a read-only FS if multipart purge is enabled

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13050

HADOOP-11694 Upgrade to AWS SDK 1.11.45

Steve Loughran Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-13047

HADOOP-11694 S3a Forward seek in stream length to be configurable

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13028

HADOOP-11694 add low level counter metrics for S3A; use in read performance tests

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13009

HADOOP-11694 add option for lazy open() on s3a

Unassigned Steve Loughran Major Resolved Invalid  
Sub-task HADOOP-12997

HADOOP-11694 s3a to pass PositionedReadable contract tests, improve readFully perf.

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-12978

HADOOP-11694 move s3a to slf4j logging

Unassigned Steve Loughran Minor Resolved Not A Problem  
Sub-task HADOOP-12977

HADOOP-11694 s3a to handle delete("/", true) robustly

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-12976

HADOOP-11694 s3a toString to be meaningful in logs

Steve Loughran Steve Loughran Trivial Resolved Duplicate  
Sub-task HADOOP-12844

HADOOP-11694 Recover when S3A fails on IOException in read()

Pieter Reuse Pieter Reuse Major Resolved Fixed  
Sub-task HADOOP-12804

HADOOP-11694 Read Proxy Password from Credential Providers in S3 FileSystem

Larry McCay Larry McCay Minor Resolved Fixed  
Sub-task HADOOP-12774

HADOOP-11694 s3a should use UGI.getCurrentUser.getShortname() for username

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-12753

HADOOP-11694 S3A JUnit tests failing if using HTTP proxy

Zoran Rajic Zoran Rajic Minor Resolved Fixed  
Sub-task HADOOP-12642

HADOOP-11694 Update documentation to cover fs.s3.buffer.dir enhancements

Unassigned Jason Archip Minor Resolved Implemented  
Sub-task HADOOP-12444

HADOOP-11694 Support lazy seek in S3AInputStream

Rajesh Balamohan Rajesh Balamohan Major Resolved Fixed  
Sub-task HADOOP-12387

HADOOP-11694 Improve S3AFastOutputStream memory management

Steve Loughran Thomas Demoor Major Resolved Duplicate  
Sub-task HADOOP-12292

HADOOP-11694 Make use of DeleteObjects optional

Thomas Demoor Thomas Demoor Major Resolved Fixed  
Sub-task HADOOP-12169

HADOOP-11694 ListStatus on empty dir in S3A lists itself instead of returning an empty list

Pieter Reuse Pieter Reuse Major Resolved Fixed  
Sub-task HADOOP-11918

HADOOP-11694 Listing an empty s3a root directory throws FileNotFound.

Lei (Eddy) Xu Lei (Eddy) Xu Minor Resolved Fixed  
Sub-task HADOOP-11874

HADOOP-11694 s3a can throw spurious IOEs on close()

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-11684

HADOOP-11694 S3a to use thread pool that blocks clients

Thomas Demoor Thomas Demoor Major Resolved Fixed  
Sub-task HADOOP-11573

HADOOP-11694 verify that s3a handles / in secret key

Steve Loughran Steve Loughran Major Resolved Done  
Sub-task HADOOP-11262

HADOOP-11694 Enable YARN to use S3A

Pieter Reuse Thomas Demoor Major Resolved Fixed  
Sub-task HADOOP-10643

HADOOP-11694 Add NativeS3Fs that delegates calls from FileContext apis to native s3 fs implementation

Sumit Kumar Sumit Kumar Major Resolved Duplicate  

Cancel