Bulk Operation

  1. Choose Issues
  2. Choose Operation
  3. Operation Details
  4. Confirmation

Step 1 of 4: Choose Issues

Cancel

T Patch Info Key Summary Assignee Reporter P Status Resolution Created Updated Due Development
Sub-task HADOOP-14838

HADOOP-13345 backport S3guard to branch-2

Steve Loughran Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-14815

HADOOP-13345 s3guard usage calls function incorrectly

Allen Wittenauer Allen Wittenauer Blocker Resolved Fixed  
Sub-task HADOOP-14810

HADOOP-13345 S3Guard: handle provisioning failure through backoff & retry (& metrics)

Unassigned Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-14809

HADOOP-13345 hadoop-aws shell profile not being built

Allen Wittenauer Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-14801

HADOOP-13345 s3guard diff demand creates a new table

Unassigned Steve Loughran Minor Resolved Duplicate  
Sub-task HADOOP-14777

HADOOP-13345 S3Guard premerge changes: java 7 build & test tuning

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-14749

HADOOP-13345 Review S3guard docs & code prior to merge

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-14735

HADOOP-13345 ITestS3AEncryptionSSEC failing in parallel s3guard runs

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-14733

HADOOP-13345 ITestS3GuardConcurrentOps failing with -Ddynamodblocal -Ds3guard

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-14645

HADOOP-13345 add a predicate/option to probe an S3A FS for being consistent

Unassigned Steve Loughran Major Resolved Won't Fix  
Sub-task HADOOP-14633

HADOOP-13345 S3Guard: optimize create codepath

Aaron Fabbri Aaron Fabbri Minor Resolved Fixed  
Sub-task HADOOP-14589

HADOOP-13345 LocalDynamoDB missing from latest AWS SDK releases

Steve Loughran Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-14548

HADOOP-13345 S3Guard: issues running parallel tests w/ S3N

Aaron Fabbri Aaron Fabbri Minor Resolved Fixed  
Sub-task HADOOP-14527

HADOOP-13345 ITestS3GuardListConsistency is too slow

Aaron Fabbri Aaron Fabbri Minor Resolved Fixed  
Sub-task HADOOP-14509

HADOOP-13345 InconsistentAmazonS3Client adds extra paths to listStatus() after delete.

Sean Mackrory Aaron Fabbri Major Resolved Duplicate  
Sub-task HADOOP-14505

HADOOP-13345 simplify mkdirs() after S3Guard delete tracking change

Sean Mackrory Aaron Fabbri Minor Resolved Fixed  
Sub-task HADOOP-14504

HADOOP-13345 ProvidedFileStatusIterator#next() may throw IndexOutOfBoundsException

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-14499

HADOOP-13345 Findbugs warning in LocalMetadataStore

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14489

HADOOP-13345 ITestS3GuardConcurrentOps requires explicit DynamoDB table name to be configured

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14488

HADOOP-13345 s3guard w/ failure injection: listStatus fails after renaming file into directory

Sean Mackrory Steve Loughran Blocker Resolved Fixed  
Sub-task HADOOP-14487

HADOOP-13345 DirListingMetadata precondition failure messages to include path at fault

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-14484

HADOOP-13345 Ensure deleted parent directory tombstones are overwritten when implicitly recreated

Sean Mackrory Sean Mackrory Major Resolved Duplicate  
Sub-task HADOOP-14476

HADOOP-13345 make InconsistentAmazonS3Client usable in downstream tests

Aaron Fabbri Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-14467

HADOOP-13345 S3Guard: Improve FNFE message when opening a stream

Aaron Fabbri Aaron Fabbri Minor Resolved Fixed  
Sub-task HADOOP-14457

HADOOP-13345 create() does not notify metadataStore of parent directories or ensure they're not existing files

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14448

HADOOP-13345 Play nice with ITestS3AEncryptionSSEC

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14433

HADOOP-13345 ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name configured

Sean Mackrory Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-14423

HADOOP-13345 s3guard will set file length to -1 on a putObjectDirect(stream, -1) call

Steve Loughran Steve Loughran Minor Resolved Fixed  
Sub-task HADOOP-14345

HADOOP-13345 S3Guard: S3GuardTool to support provisioning existing metadata store

Steve Loughran Mingliang Liu Minor Resolved Duplicate  
Sub-task HADOOP-14323

HADOOP-13345 ITestS3GuardListConsistency failure w/ Local, authoritative metadata store

Aaron Fabbri Aaron Fabbri Minor Resolved Fixed  
Sub-task HADOOP-14288

HADOOP-13345 TestDynamoDBMetadataStore is broken unless we can fail faster without a table version

Sean Mackrory Sean Mackrory Critical Resolved Fixed  
Sub-task HADOOP-14282

HADOOP-13345 S3Guard: DynamoDBMetadata::prune() should self interrupt correctly

Mingliang Liu Mingliang Liu Minor Resolved Fixed  
Sub-task HADOOP-14266

HADOOP-13345 S3Guard: S3AFileSystem::listFiles() to employ MetadataStore

Mingliang Liu Mingliang Liu Major Resolved Fixed  
Sub-task HADOOP-14263

HADOOP-13345 TestS3GuardTool hangs/fails when offline: it's an IT test

Mingliang Liu Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-14236

HADOOP-13345 S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

Mingliang Liu Mingliang Liu Critical Resolved Fixed  
Sub-task HADOOP-14227

HADOOP-13345 S3Guard: ITestS3AConcurrentOps is not cleaning up test data

Mingliang Liu Mingliang Liu Minor Resolved Fixed  
Sub-task HADOOP-14226

HADOOP-13345 S3Guard: DynamoDBMetadataStore::move() should populate ancestor directories of destination paths

Mingliang Liu Mingliang Liu Major Resolved Fixed  
Sub-task HADOOP-14221

HADOOP-13345 Add s3guardtool dump command

Unassigned Steve Loughran Minor Resolved Works for Me  
Sub-task HADOOP-14215

HADOOP-13345 DynamoDB client should waitForActive on existing tables

Sean Mackrory Sean Mackrory Critical Resolved Fixed  
Sub-task HADOOP-14181

HADOOP-13345 Add validation of DynamoDB region

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14172

HADOOP-13345 S3Guard: import does not import empty directory

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14168

HADOOP-13345 S3GuardTool tests should not run if S3Guard is not set up

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14165

HADOOP-13345 Add S3Guard.dirListingUnion in S3AFileSystem#listFiles, listLocatedStatus

Unassigned Rajesh Balamohan Minor Closed Duplicate  
Sub-task HADOOP-14145

HADOOP-13345 Ensure GenericOptionParser is used for S3Guard CLI

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14144

HADOOP-13345 s3guard: CLI diff non-empty after import on new table

Sean Mackrory Aaron Fabbri Minor Resolved Fixed  
Sub-task HADOOP-14130

HADOOP-13345 Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients

Mingliang Liu Mingliang Liu Major Resolved Fixed  
Sub-task HADOOP-14129

HADOOP-13345 ITestS3ACredentialsInURL sometimes fails

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14125

HADOOP-13345 s3guard tool tests aren't isolated; can't run in parallel

Sean Mackrory Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-14110

HADOOP-13345 In S3AFileSystem, make getAmazonClient() package private; export getBucketLocation()

Steve Loughran Steve Loughran Critical Resolved Fixed  
Sub-task HADOOP-14107

HADOOP-13345 ITestS3GuardListConsistency fails intermittently

Mingliang Liu Mingliang Liu Minor Resolved Fixed  
Sub-task HADOOP-14096

HADOOP-13345 s3guard: regression in dirListingUnion

Aaron Fabbri Aaron Fabbri Critical Resolved Fixed  
Sub-task HADOOP-14094

HADOOP-13345 Rethink S3GuardTool options

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14090

HADOOP-13345 Allow users to specify region for DynamoDB table instead of endpoint

Sean Mackrory Sean Mackrory Major Resolved Won't Fix  
Sub-task HADOOP-14085

HADOOP-13345 Drop unnecessary type assertion and cast

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14079

HADOOP-13345 Fix breaking link in s3guard.md

Mingliang Liu Mingliang Liu Trivial Resolved Fixed  
Sub-task HADOOP-14051

HADOOP-13345 S3Guard: link docs from index, fix typos

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-14046

HADOOP-13345 Metastore destruction test creates table without version marker

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14041

HADOOP-13345 CLI command to prune old metadata

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14036

HADOOP-13345 S3Guard: intermittent duplicate item keys failure

Mingliang Liu Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-14027

HADOOP-13345 Implicitly creating DynamoDB table ignores endpoint config

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14023

HADOOP-13345 S3Guard: DynamoDBMetadataStore logs nonsense region

Sean Mackrory Sean Mackrory Minor Resolved Resolved  
Sub-task HADOOP-14020

HADOOP-13345 Optimize dirListingUnion

Sean Mackrory Sean Mackrory Major Resolved Fixed  
Sub-task HADOOP-14013

HADOOP-13345 S3Guard: fix multi-bucket integration tests

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-14012

HADOOP-13345 Handled dynamo exceptions in translateException

Unassigned Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-14007

HADOOP-13345 cli to list info about a bucket (S3guard or not)

Unassigned Steve Loughran Major Resolved Duplicate  
Sub-task HADOOP-13998

HADOOP-13345 Merge initial S3guard release into trunk

Steve Loughran Steve Loughran Major Resolved Done  
Sub-task HADOOP-13995

HADOOP-13345 s3guard cli: make tests easier to run and address failure

Sean Mackrory Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13985

HADOOP-13345 s3guard: add a version marker to every table

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13981

HADOOP-13345 S3Guard CLI: Add documentation

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13966

HADOOP-13345 Add ability to start DDB local server in every test

Mingliang Liu Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13960

HADOOP-13345 Initialize DynamoDBMetadataStore without associated S3AFileSystem

Mingliang Liu Mingliang Liu Major Resolved Fixed  
Sub-task HADOOP-13959

HADOOP-13345 S3guard: replace dynamo.describe() call in init with more efficient query

Mingliang Liu Steve Loughran Minor Closed Fixed  
Sub-task HADOOP-13937

HADOOP-13345 Mock bucket locations in MockS3ClientFactory

Mingliang Liu Mingliang Liu Minor Resolved Fixed  
Sub-task HADOOP-13934

HADOOP-13345 S3Guard: DynamoDBMetadataStore#move() could be throwing exception due to BatchWriteItem limits

Mingliang Liu Rajesh Balamohan Minor Resolved Fixed  
Sub-task HADOOP-13931

HADOOP-13345 S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()

Mingliang Liu Rajesh Balamohan Minor Resolved Fixed  
Sub-task HADOOP-13926

HADOOP-13345 S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

Mingliang Liu Rajesh Balamohan Major Resolved Fixed  
Sub-task HADOOP-13925

HADOOP-13345 S3Guard: NPE when table is already populated in dynamodb and user specifies "fs.s3a.s3guard.ddb.table.create=false"

Mingliang Liu Rajesh Balamohan Minor Closed Duplicate  
Sub-task HADOOP-13914

HADOOP-13345 s3guard: improve S3AFileStatus#isEmptyDirectory handling

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13908

HADOOP-13345 S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore

Mingliang Liu Mingliang Liu Major Resolved Fixed  
Sub-task HADOOP-13904

HADOOP-13345 DynamoDBMetadataStore to handle DDB throttling failures through retry policy

Aaron Fabbri Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13899

HADOOP-13345 tune dynamodb client & tests

Steve Loughran Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13893

HADOOP-13345 dynamodb dependency -> compile

Mingliang Liu Steve Loughran Major Resolved Fixed  
Sub-task HADOOP-13886

HADOOP-13345 s3guard: ITestS3AFileOperationCost.testFakeDirectoryDeletion failure

Mingliang Liu Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13877

HADOOP-13345 S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13876

HADOOP-13345 S3Guard: better support for multi-bucket access

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13850

HADOOP-13345 s3guard to log choice of metadata store at debug

Mingliang Liu Steve Loughran Trivial Resolved Fixed  
Sub-task HADOOP-13793

HADOOP-13345 s3guard: add inconsistency injection, integration tests

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13760

HADOOP-13345 S3Guard: add delete tracking

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13736

HADOOP-13345 Change PathMetadata to hold S3AFileStatus instead of FileStatus.

Lei (Eddy) Xu Lei (Eddy) Xu Major Resolved Won't Fix  
Sub-task HADOOP-13651

HADOOP-13345 S3Guard: S3AFileSystem Integration with MetadataStore

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13650

HADOOP-13345 S3Guard: Provide command line tools to manipulate metadata store.

Lei (Eddy) Xu Lei (Eddy) Xu Major Resolved Fixed  
Sub-task HADOOP-13631

HADOOP-13345 S3Guard: implement move() for LocalMetadataStore, add unit tests

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13589

HADOOP-13345 S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

Steve Loughran Chris Nauroth Major Resolved Fixed  
Sub-task HADOOP-13573

HADOOP-13345 S3Guard: create basic contract tests for MetadataStore implementations

Aaron Fabbri Aaron Fabbri Major Resolved Fixed  
Sub-task HADOOP-13455

HADOOP-13345 S3Guard: Write end user docs, change table autocreate default.

Aaron Fabbri Chris Nauroth Major Resolved Fixed  
Sub-task HADOOP-13453

HADOOP-13345 S3Guard: Instrument new functionality with Hadoop metrics.

Ai Deng Chris Nauroth Major Resolved Fixed  
Sub-task HADOOP-13452

HADOOP-13345 S3Guard: Implement access policy for intra-client consistency with in-memory metadata store.

Aaron Fabbri Chris Nauroth Major Resolved Fixed  
Sub-task HADOOP-13451

HADOOP-13345 S3Guard: Implement access policy using metadata store as source of truth.

Unassigned Chris Nauroth Major Closed Invalid  
Sub-task HADOOP-13450

HADOOP-13345 S3Guard: Implement access policy providing strong consistency with S3 as source of truth.

Unassigned Chris Nauroth Major Closed Invalid  
Sub-task HADOOP-13449

HADOOP-13345 S3Guard: Implement DynamoDBMetadataStore.

Mingliang Liu Chris Nauroth Major Resolved Fixed  
Sub-task HADOOP-13448

HADOOP-13345 S3Guard: Define MetadataStore interface.

Chris Nauroth Chris Nauroth Major Resolved Fixed  
Sub-task HADOOP-13446

HADOOP-13345 Support running isolated unit tests separate from AWS integration tests.

Chris Nauroth Chris Nauroth Major Resolved Fixed  

Cancel