Bulk Operation

  1. Choose Issues
  2. Choose Operation
  3. Operation Details
  4. Confirmation

Step 1 of 4: Choose Issues

Cancel

T Patch Info Key Summary Assignee Reporter P Status Resolution Created Updated Due Development
Sub-task HADOOP-16667

HADOOP-15620 s3guard LimitExceededException -too many tables

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16666

HADOOP-15620 typo in TestNeworkBinding

Steve Loughran Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16653

HADOOP-15620 S3Guard DDB overreacts to no tag access

Gabor Bota Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-16651

HADOOP-15620 S3 getBucketLocation() can return "US" for us-east

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16650

HADOOP-15620 ITestS3AClosedFS failing -junit test thread

Steve Loughran Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-16645

HADOOP-15620 S3A Delegation Token extension point to use StoreContext

Steve Loughran Steve Loughran Minor Patch Available Unresolved  
Sub-task HADOOP-16644

HADOOP-15620 Retrive modtime of PUT file from store, via response or HEAD

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16635

HADOOP-15620 S3A innerGetFileStatus s"directories only" scan still does a HEAD

Steve Loughran Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-16634

HADOOP-15620 S3A ITest failures without S3Guard

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16633

HADOOP-15620 Tune hadoop-aws parallel test surefire/failsafe settings

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16632

HADOOP-15620 Speculating & Partitioned S3A magic committers can leave pending files under __magic

Steve Loughran Steve Loughran Minor Patch Available Unresolved  
Sub-task HADOOP-16626

HADOOP-15620 S3A ITestRestrictedReadAccess fails

Steve Loughran Siddharth Seth Major Resolved Fixed  
Sub-task HADOOP-16618

HADOOP-15620 increase the default number of threads and http connections in S3A

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16613

HADOOP-15620 s3a to set fake directory marker contentType to application/x-directory

Unassigned Jose Torres Minor Open Unresolved  
Sub-task HADOOP-16607

HADOOP-15620 s3a attempts to look up password/encryption fail if JCEKS file unreadable

Unassigned Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-16591

HADOOP-15620 S3A ITest*MRjob failures

Siddharth Seth Siddharth Seth Major Resolved Fixed  
Sub-task HADOOP-16575

HADOOP-15620 ITestS3ARemoteFileChanged tests fail if you set the bucket to versionid tracking

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16574

HADOOP-15620 ITestS3AAWSCredentialsProvider tests fail if a bucket has DTs enabled

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16573

HADOOP-15620 IAM role created by S3A DT doesn't include DynamoDB scan

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16572

HADOOP-15620 S3A DT support to warn when loading expired token

Steve Loughran Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16571

HADOOP-15620 s3a to improve diags on s3a bad request message

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16570

HADOOP-15620 S3A committers leak threads/raises OOM on job/task commit at scale

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16568

HADOOP-15620 S3A FullCredentialsTokenBinding fails if local credentials are unset

Steve Loughran Steve Loughran Major In Progress Unresolved  
Sub-task HADOOP-16567

HADOOP-15620 S3A Secret access to fall back to XML if credential provider raises IOE.

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16546

HADOOP-15620 make sure staging committers collect DTs for the staging FS

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16538

HADOOP-15620 S3AFilesystem trash handling should respect the current UGI

Unassigned Siddharth Seth Major Open Unresolved  
Sub-task HADOOP-16522

HADOOP-15620 Encrypt S3A buffered data on disk

Unassigned Mike Yoder Major Open Unresolved  
Sub-task HADOOP-16500

HADOOP-15620 S3ADelegationTokens to only log at debug on startup

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16499

HADOOP-15620 S3A retry policy to be exponential

Steve Loughran Steve Loughran Critical Resolved Fixed  
Sub-task HADOOP-16493

HADOOP-15620 S3AFilesystem.initiateRename() can skip check on dest.parent status if src has same parent

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16482

HADOOP-15620 S3A doesn't actually verify paths have the correct authority

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16480

HADOOP-15620 S3 Select Exceptions are not being converted to IOEs

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16478

HADOOP-15620 S3Guard bucket-info fails if the bucket location is denied to the caller

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16477

HADOOP-15620 S3A delegation token tests fail if fs.s3a.encryption.key set

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16471

HADOOP-15620 Restore (documented) fs.s3a.SharedInstanceProfileCredentialsProvider

Steve Loughran Steve Loughran Major Resolved Done  
Sub-task HADOOP-16470

HADOOP-15620 Make last AWS credential provider in default auth chain EC2ContainerCredentialsProviderWrapper

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16469

HADOOP-15620 Typo in s3a committers.md doc

Unassigned Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-16468

HADOOP-15620 S3AFileSystem.getContentSummary() to use listFiles(recursive)

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16465

HADOOP-15620 S3AFileSystem.listLocatedStatus to LIST before HEAD

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16458

HADOOP-15620 LocatedFileStatusFetcher scans failing intermittently against S3 store

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16450

HADOOP-15620 ITestS3ACommitterFactory failing, S3 client is not inconsistent

Steve Loughran Steve Loughran Trivial Open Unresolved  
Sub-task HADOOP-16432

HADOOP-15620 Review S3A documentation to make sure it is consistent with the current codebase

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16427

HADOOP-15620 Downgrade INFO message on rm s3a root dir to DEBUG

Unassigned Steve Loughran Trivial Resolved Won't Fix  
Sub-task HADOOP-16422

HADOOP-15620 AbstractITCommitMRJob.testMRJob test failures

Unassigned Gabor Bota Major Resolved Duplicate  
Sub-task HADOOP-16420

HADOOP-15620 S3A returns 400 "bad request" on a single path within an S3 bucket

Unassigned Steve Loughran Minor Resolved Cannot Reproduce  
Sub-task HADOOP-16415

HADOOP-15620 Speed up S3A test runs

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16414

HADOOP-15620 ITestS3AMiniYarnCluster fails on sequential runs with Kerberos error

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16413

HADOOP-15620 ITestS3ARemoteFileChanged doesn't overwrite test data creation

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16407

HADOOP-15620 Improve isolation of FS instances in S3A committer tests

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16387

HADOOP-15620 S3A openFile() options to allow etag/version to be set

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16379

HADOOP-15620 S3AInputStream#unbuffer should merge input stream stats into fs-wide stats

Sahil Takiar Sahil Takiar Major Resolved Fixed  
Sub-task HADOOP-16371

HADOOP-15620 Option to disable GCM for SSL connections when running on Java 8

Sahil Takiar Sahil Takiar Major Resolved Fixed  
Sub-task HADOOP-16360

HADOOP-15620 S3A NullPointerException: null uri host. This can be caused by unencoded / in the password string

Unassigned Ladislav Jech Minor Resolved Won't Fix  
Sub-task HADOOP-16357

HADOOP-15620 TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists

Steve Loughran Prabhu Joseph Minor Resolved Fixed  
Sub-task HADOOP-16347

HADOOP-15620 MapReduce job tasks fails on S3A ssl3_get_server_certificate:certificate verify

Steve Loughran Prabhu Joseph Major Resolved Cannot Reproduce  
Sub-task HADOOP-16346

HADOOP-15620 Stabilize S3A OpenSSL support

Unassigned Steve Loughran Blocker Open Unresolved  
Sub-task HADOOP-16333

HADOOP-15620 Test Hang in S3A S3guard test MetadataStoreTestBase.testListChildren

Unassigned Steve Loughran Major Resolved Cannot Reproduce  
Sub-task HADOOP-16332

HADOOP-15620 Remove S3A's depedency on http core

Steve Loughran Steve Loughran Critical Resolved Fixed  
Sub-task HADOOP-16330

HADOOP-15620 Regression: TestStagingPartitionedJobCommit failing with empty etag list

Steve Loughran Steve Loughran Minor Resolved Done  
Sub-task HADOOP-16328

HADOOP-15620 ClassCastException in S3GuardTool.checkMetadataStoreUri

Steve Loughran Steve Loughran Major Resolved Cannot Reproduce  
Sub-task HADOOP-16327

HADOOP-15620 s3a test docs to mention non-auth; or s3a tests to default to non-auth

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16324

HADOOP-15620 S3A Delegation Token code to spell "Marshalled" as Marshaled

Steve Loughran Steve Loughran Blocker Resolved Won't Fix  
Sub-task HADOOP-16319

HADOOP-15620 ITestS3AMiscOperations.testEmptyFileChecksums and ITestS3AMiscOperations.testNonEmptyFileChecksumsUnencrypted fail with default encryption enabled on bucket

Unassigned Ben Roling Major Open Unresolved  
Sub-task HADOOP-16316

HADOOP-15620 S3A delegation tests fail if you set fs.s3a.secret.key

Unassigned Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-16313

HADOOP-15620 multipart/huge file upload tests to look at checksums returned

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16308

HADOOP-15620 ITestS3AContractSeek teardown closes test FS before superclass can do its cleanup

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16233

HADOOP-15620 S3AFileStatus to declare that isEncrypted() is always true

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-16223

HADOOP-15620 remove misleading fs.s3a.delegation.tokens.enabled prompt

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16207

HADOOP-15620 Improved S3A MR tests

Steve Loughran Steve Loughran Critical Resolved Fixed  
Sub-task HADOOP-16203

HADOOP-15620 ITestS3AContractGetFileStatusV1List may have consistency issues

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16202

HADOOP-15620 S3A openFile() operation to support explicit versionID, etag parameters

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16201

HADOOP-15620 S3AFileSystem#innerMkdirs builds needless lists

Lokesh Jain Lokesh Jain Trivial Resolved Fixed  
Sub-task HADOOP-16197

HADOOP-15620 S3AUtils.translateException to map CredentialInitializationException to AccessDeniedException

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16195

HADOOP-15620 S3A MarshalledCredentials.toString() doesn't print full date/time of expiry

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-16193

HADOOP-15620 add extra S3A MPU test to see what happens if a file is created during the MPU

Steve Loughran Steve Loughran Minor Reopened Unresolved  
Sub-task HADOOP-16190

HADOOP-15620 S3A copyFile operation to include source versionID or etag in the copy request

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-16189

HADOOP-15620 S3A copy/rename of large files to be parallelized as a multipart operation

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16188

HADOOP-15620 s3a rename failed during copy, "Unable to copy part" + 200 error code

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16177

HADOOP-15620 regression: ITestS3AMiniYarnCluster failing on branch-3.2

Unassigned Steve Loughran Major Resolved Invalid  
Sub-task HADOOP-16176

HADOOP-15620 Add some tests about S3 timestamp tracking

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16175

HADOOP-15620 S3Guard to add DynamoDBLocal Support

lqjacklee lqjacklee Major Resolved Won't Fix  
Sub-task HADOOP-16164

HADOOP-15620 S3aDelegationTokens to add accessor for tests to get at the token binding

Unassigned Steve Loughran Critical Open Unresolved  
Sub-task HADOOP-16155

HADOOP-15620 S3AInputStream read(bytes[]) to not retry on read failure: pass action up

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16154

HADOOP-15620 ITestS3A select tests fail if user kinited in

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16134

HADOOP-15620 Add S3AWriteOpContext for write ops; pass in statistics and other settings

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-16133

HADOOP-15620 S3A statistic collection underrecords bytes written in helper threads

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16132

HADOOP-15620 Support multipart download in S3AFileSystem

Unassigned Justin Uang Major Open Unresolved  
Sub-task HADOOP-16128

HADOOP-15620 Some S3A tests leak filesystem instances

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16124

HADOOP-15620 Extend documentation in testing.md about endpoint constants

Adam Antal Adam Antal Trivial Resolved Fixed  
Sub-task HADOOP-16117

HADOOP-15620 Update AWS SDK to 1.11.563

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16109

HADOOP-15620 Parquet reading S3AFileSystem causes EOF

Steve Loughran Dave Christianson Blocker Resolved Fixed  
Sub-task HADOOP-16101

HADOOP-15620 Use lighter-weight alternatives to innerGetFileStatus where possible

Unassigned Sean Mackrory Major Open Unresolved  
Sub-task HADOOP-16093

HADOOP-15620 Move DurationInfo from hadoop-aws to hadoop-common org.apache.hadoop.util

Abhishek Modi Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-16090

HADOOP-15620 S3A Client to add explicit support for versioned stores

Steve Loughran Dmitri Chmelev Minor Patch Available Unresolved  
Sub-task HADOOP-16079

HADOOP-15620 Token.toString faulting if any token listed can't load.

Steve Loughran Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-16066

HADOOP-15620 S3a DelegationToken bindings to to support a "correlation ID" for the UA header

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16058

HADOOP-15620 S3A tests to include Terasort

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-16050

HADOOP-15620 s3a SSL connections should use OpenSSL

Sahil Takiar Justin Uang Major Resolved Fixed  
Sub-task HADOOP-16033

HADOOP-15620 hamcrest-library declaration in hadoop-aws to be scoped test

Steve Loughran Steve Loughran Blocker Resolved Duplicate  
Sub-task HADOOP-16027

HADOOP-15620 [DOC] Effective use of FS instances during S3A integration tests

Gabor Bota Gabor Bota Major Resolved Fixed  
Sub-task HADOOP-16017

HADOOP-15620 Add some S3A-specific create file options

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-16015

HADOOP-15620 Add bouncycastle jars to hadoop-aws as test dependencies

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-15990

HADOOP-15620 S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2

lqjacklee Steve Loughran Major Patch Available Unresolved  
Sub-task HADOOP-15964

HADOOP-15620 Add S3A support for Async Scatter/Gather IO

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15961

HADOOP-15620 S3A committers: make sure there's regular progress() calls

lqjacklee Steve Loughran Minor Patch Available Unresolved  
Sub-task HADOOP-15955

HADOOP-15620 declare that fs.s3a.ext. is a prefix for arbitrary extensions

Unassigned Steve Loughran Trivial Open Unresolved  
Sub-task HADOOP-15944

HADOOP-15620 S3AInputStream logging to make it easier to debug file leakage

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15932

HADOOP-15620 Oozie unable to create sharelib in s3a filesystem

Steve Loughran Soumitra Sulav Critical Resolved Fixed  
Sub-task HADOOP-15924

HADOOP-15620 Hadoop aws does not use shaded jars

Unassigned Bharat Viswanadham Major Resolved Won't Fix  
Sub-task HADOOP-15920

HADOOP-15620 get patch for S3a nextReadPos(), through Yetus

lqjacklee Steve Loughran Major Resolved Done  
Sub-task HADOOP-15915

HADOOP-15620 Report problems w/ local S3A buffer directory meaningfully

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15890

HADOOP-15620 Some S3A committer tests don't match ITest* pattern; don't run in maven

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-15875

HADOOP-15620 S3AInputStream.seek should throw EOFException if seeking past the end of file

Unassigned Shixiong Zhu Minor Open Unresolved  
Sub-task HADOOP-15870

HADOOP-15620 S3AInputStream.remainingInFile should use nextReadPos

lqjacklee Shixiong Zhu Major Reopened Unresolved  
Sub-task HADOOP-15848

HADOOP-15620 ITestS3AContractMultipartUploader#testMultipartUploadEmptyPart test error

Ewan Higgs Gabor Bota Major Resolved Fixed  
Sub-task HADOOP-15839

HADOOP-15620 Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-15826

HADOOP-15620 @Retries annotation of putObject() call & uses wrong

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-15807

HADOOP-15620 ITestS3AContractRootDir failure on non-S3Guarded bucket

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15806

HADOOP-15620 Move ITestS3AMiniYarnCluster to S3A committers

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-15782

HADOOP-15620 Clarify committers.md around v2 failure handling

Unassigned Gera Shegalov Major Open Unresolved  
Sub-task HADOOP-15751

HADOOP-15620 AWS Data read stack trace in S3a putObjectDirect

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15750

HADOOP-15620 remove obsolete S3A test ITestS3ACredentialsInURL

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-15748

HADOOP-15620 S3 listing inconsistency can raise NPE in globber

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-15747

HADOOP-15620 warning about user:pass in URI to explicitly call out Hadoop 3.2 as removal

Steve Loughran Steve Loughran Major Resolved Won't Fix  
Sub-task HADOOP-15737

HADOOP-15620 intermittent failure of ITestS3GuardListConsistency.testInconsistentS3ClientDeletes in parallel runs

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15729

HADOOP-15620 [s3a] stop treat fs.s3a.max.threads as the long-term minimum

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-15672

HADOOP-15620 add s3guard CLI command to generate session keys for an assumed role

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-15667

HADOOP-15620 FileSystemMultipartUploader should verify that UploadHandle has non-0 length

Ewan Higgs Ewan Higgs Major Resolved Fixed  
Sub-task HADOOP-15658

HADOOP-15620 Memory leak in S3AOutputStream

Steve Loughran Piotr Nowojski Major Resolved Fixed  
Sub-task HADOOP-15651

HADOOP-15620 NPE in S3AInputStream.read() in ITestS3AInconsistency.testOpenFailOnRead

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15650

HADOOP-15620 Add custom InstanceProfileCredentialsProvider with more resilience to throttling

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15631

HADOOP-15620 Remove transient dependency on hadoop-hdfs-client

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15628

HADOOP-15620 S3A Filesystem does not check return from AmazonS3Client deleteObjects

Unassigned Steve Jacobs Minor Open Unresolved  
Sub-task HADOOP-15625

HADOOP-15620 S3A input stream to use etags/version number to detect changed source files

Ben Roling Brahma Reddy Battula Major Resolved Fixed  
Sub-task HADOOP-15604

HADOOP-15620 Bulk commits of S3A MPUs place needless excessive load on S3 & S3Guard

Steve Loughran Gabor Bota Major Resolved Fixed  
Sub-task HADOOP-15603

HADOOP-15620 S3A to support configuring various AWS S3 client extended options

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15575

HADOOP-15620 Add a way for an FS instance to say "really, no trash interval at all"

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15570

HADOOP-15620 s3guard uploads command to list date and initiator of outstanding uploads

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15555

HADOOP-15620 S3AFileStatus to add a serialVersionUID; review & test serialization

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15503

HADOOP-15620 strip s3.amazonaws.com off hostnames before making s3a calls

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15479

HADOOP-15620 s3guard bucket-info command to add a verify-property <key>=<value> <bucket>

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15460

HADOOP-15620 S3A FS to add "s3a:no-existence-checks" to the builder file creation option set

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15421

HADOOP-15620 Stabilise/formalise the JSON _SUCCESS format used in the S3A committers

Unassigned Steve Loughran Major Resolved Won't Fix  
Sub-task HADOOP-15409

HADOOP-15620 S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2

lqjacklee Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-15370

HADOOP-15620 S3A log message on rm s3a://bucket/ not intuitive

Gabor Bota Steve Loughran Trivial Resolved Fixed  
Sub-task HADOOP-15364

HADOOP-15620 Add support for S3 Select to S3A

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-15348

HADOOP-15620 S3A Input Stream bytes read counter isn't getting through to StorageStatistics/instrumentation properly

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15347

HADOOP-15620 S3ARetryPolicy to handle AWS 500 responses/error code TooBusyException with the throttle backoff policy

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15303

HADOOP-15620 make s3a read fault injection configurable including "off"

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15249

HADOOP-15620 review S3A translateException translation matches IBM CORS spec

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15245

HADOOP-15620 S3AInputStream.skip() to use lazy seek

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-15243

HADOOP-15620 test and document use of fs.s3a.signing-algorithm

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15228

HADOOP-15620 S3A Retry policy to retry on NoResponseException

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-15224

HADOOP-15620 builld up md5 checksum as blocks are built in S3ABlockOutputStream; validate upload

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15135

HADOOP-15620 S3a to support get/set permissions through S3 object tags

Unassigned Steve Loughran Major Resolved Won't Fix  
Sub-task HADOOP-15091

HADOOP-15620 S3aUtils.getEncryptionAlgorithm() always logs@Debug "Using SSE-C"

Unassigned Steve Loughran Trivial Resolved Duplicate  
Sub-task HADOOP-15069

HADOOP-15620 support git-secrets commit hook to keep AWS secrets out of git

Steve Loughran Steve Loughran Minor Patch Available Unresolved  
Sub-task HADOOP-15057

HADOOP-15620 s3guard bucket-info command to include default bucket encryption info

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-15000

HADOOP-15620 s3a new getdefaultblocksize be called in getFileStatus which has not been implemented in s3afilesystem yet

Unassigned Yonger Minor Open Unresolved  
Sub-task HADOOP-14972

HADOOP-15620 S3A add histogram metrics types for latency, etc.

Sean Mackrory Sean Mackrory Major Open Unresolved  
Sub-task HADOOP-14943

HADOOP-15620 Add common getFileBlockLocations() emulation for object stores, including S3A

Steve Loughran Steve Loughran Major Patch Available Unresolved  
Sub-task HADOOP-14937

HADOOP-15620 initial part uploads seem to block unnecessarily in S3ABlockOutputStream

Steven Rand Steven Rand Major Open Unresolved  
Sub-task HADOOP-14837

HADOOP-15620 Handle S3A "glacier" data

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14776

HADOOP-15620 clean up ITestS3AFileSystemContract

Unassigned Steve Loughran Minor Patch Available Unresolved  
Sub-task HADOOP-14762

HADOOP-15620 S3A warning of obsolete encryption key which is never used

Unassigned Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-14750

HADOOP-15620 s3guard to provide better diags on ddb init failures

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14747

HADOOP-15620 S3AInputStream to implement CanUnbuffer

Sahil Takiar Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-14707

HADOOP-15620 AbstractContractDistCpTest to test attr preservation with -p, verify blobstores downgrade

Steve Loughran Steve Loughran Major Open Unresolved  
Sub-task HADOOP-14661

HADOOP-15620 S3A to support Requester Pays Buckets

Mandus Momberg Mandus Momberg Minor Patch Available Unresolved  
Sub-task HADOOP-14639

HADOOP-15620 test YARN log collection works to s3a

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14621

HADOOP-15620 S3A client raising ConnectionPoolTimeoutException in test

Unassigned Steve Loughran Minor Resolved Cannot Reproduce  
Sub-task HADOOP-14619

HADOOP-15620 S3A authenticators to log origin of .secret.key options

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14603

HADOOP-15620 S3A input stream to support ByteBufferReadable

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14564

HADOOP-15620 s3a test can hang in teardown with network problems

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14556

HADOOP-15620 S3A to support Delegation Tokens

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-14551

HADOOP-15620 S3A init hangs if you try to connect while the system is offline

Unassigned Steve Loughran Minor Resolved Won't Fix  
Sub-task HADOOP-14510

HADOOP-15620 Use error code detail in AWS server responses for finer grained exceptions

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-14391

HADOOP-15620 s3a: auto-detect region for bucket and use right endpoint

Aaron Fabbri Aaron Fabbri Major Open Unresolved  
Sub-task HADOOP-14334

HADOOP-15620 S3 SSEC tests to downgrade when running against a mandatory encryption object store

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14302

HADOOP-15620 Test MR split optimisation with recursive listing

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-14237

HADOOP-15620 S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes

Kazuyuki Tanimura Kazuyuki Tanimura Major Patch Available Unresolved  
Sub-task HADOOP-14132

HADOOP-15620 Filesystem discovery to stop loading implementation classes

Steve Loughran Steve Loughran Major Open Unresolved  
Sub-task HADOOP-14124

HADOOP-15620 S3AFileSystem silently deletes "fake" directories when writing a file.

Unassigned Joel Baranick Minor Resolved Won't Fix  
Sub-task HADOOP-14093

HADOOP-15620 add tests/docs for HAR files on s3a

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-14070

HADOOP-15620 S3a: Failed to reset the request input stream/make S3A uploadPart() retriable

Unassigned Seth Fitzsimmons Major Open Unresolved  
Sub-task HADOOP-13887

HADOOP-15620 Encrypt S3A data client-side with AWS SDK

Igor Mazur Jeeyoung Kim Minor Patch Available Unresolved  
Sub-task HADOOP-13884

HADOOP-15620 s3a create(overwrite=true) to only look for dir/ and list entries, not file

Steve Loughran Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-13873

HADOOP-15620 log DNS addresses on s3a init

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13846

HADOOP-15620 S3A to implement rename(final Path src, final Path dst, final Rename... options)

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13845

HADOOP-15620 s3a to instrument duration of HTTP calls

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13841

HADOOP-15620 S3a auth exception to link to a wiki page on the problem

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13762

HADOOP-15620 S3A: Set thread names with more specific information about the call.

Unassigned Chris Nauroth Major Open Unresolved  
Sub-task HADOOP-13728

HADOOP-15620 S3A can support short user-friendly aliases for configuration of credential providers.

Unassigned Chris Nauroth Minor Open Unresolved  
Sub-task HADOOP-13704

HADOOP-15620 S3A getContentSummary() to move to listFiles(recursive) to count children; instrument use

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13695

HADOOP-15620 S3A to use a thread pool for async path operations

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13656

HADOOP-15620 fs -expunge to take a filesystem

Shweta Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13637

HADOOP-15620 improve setting of max connections in AWS client

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13636

HADOOP-15620 s3a rm on the CLI generates deprecation warning on io.bytes.per.checksum

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13611

HADOOP-15620 FileSystem/s3a processDeleteOnExit to skip the exists() check

Unassigned Steve Loughran Trivial Open Unresolved  
Sub-task HADOOP-13600

HADOOP-15620 S3a rename() to copy files in a directory in parallel

Unassigned Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13585

HADOOP-15620 shell rm command to not rename to ~/.Trash in object stores

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13551

HADOOP-15620 hook up AwsSdkMetrics to hadoop metrics

Sean Mackrory Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13525

HADOOP-15620 Optimize uses of FS operations in the ASF analysis frameworks and libraries

Steve Loughran Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13507

HADOOP-15620 export s3a BlockingThreadPoolExecutorService pool info (size, load) as metrics

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13432

HADOOP-15620 S3A: Consider using TransferManager.download for copyToLocalFile

Unassigned Rajesh Balamohan Minor Resolved Won't Fix  
Sub-task HADOOP-13430

HADOOP-15620 Optimize getFileStatus in S3A

Steven K. Wong Steven K. Wong Minor Open Unresolved  
Sub-task HADOOP-13407

HADOOP-15620 s3a directory housekeeping operations to be done in async thread

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13402

HADOOP-15620 S3A should allow renaming to a pre-existing destination directory to move the source path under that directory, similar to HDFS.

Unassigned Rajesh Balamohan Minor Resolved Won't Fix  
Sub-task HADOOP-13373

HADOOP-15620 Add S3A implementation of FSMainOperationsBaseTest

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-13334

HADOOP-15620 AmazonClient 30x exceptions to include redirect URL in message

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13330

HADOOP-15620 Parallelize S3A directory deletes

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13294

HADOOP-15620 Test hadoop fs shell against s3a; fix problems

Unassigned Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13293

HADOOP-15620 add a special 0 byte input stream for empty blobs

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13276

HADOOP-15620 S3a operations keep retrying if the password is wrong

Thomas Poepping Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-13221

HADOOP-15620 s3a create() doesn't check for an ancestor path being a file

Sean Mackrory Steve Loughran Major Open Unresolved  
Sub-task HADOOP-13007

HADOOP-15620 cherry pick s3 ehancements from PrestoS3FileSystem

Unassigned Steve Loughran Minor Open Unresolved  
Sub-task HADOOP-12949

HADOOP-15620 Add HTrace to the s3a connector

Madhawa Kasun Gunasekara Madhawa Gunasekara Major Resolved Won't Fix  
Sub-task HADOOP-12878

HADOOP-15620 Impersonate hosts in s3a for better data locality handling

Thomas Demoor Thomas Demoor Major Open Unresolved  
Sub-task HADOOP-12020

HADOOP-15620 Support AWS S3 reduced redundancy storage class

Unassigned Yann Landrin-Schweitzer Major Open Unresolved  
Sub-task HADOOP-11621

HADOOP-15620 s3a doesn't consider blobs with trailing / and content-length >0 as directories

Unassigned Denis Jannot Major Open Unresolved  

Cancel