Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13345 S3Guard: Improved Consistency for S3A
  3. HADOOP-13998

Merge initial S3guard release into trunk

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Done
    • 3.0.0-beta1
    • 3.0.0-beta1
    • fs/s3
    • None

    Description

      JIRA to link in all the things we think are needed for a preview/merge into trunk

      Attachments

        1. HADOOP-13998-005.patch
          592 kB
          Steve Loughran
        2. HADOOP-13998-004.patch
          592 kB
          Steve Loughran
        3. HADOOP-13998-003.patch
          591 kB
          Steve Loughran
        4. HADOOP-13998-002.patch
          590 kB
          Steve Loughran
        5. HADOOP-13998-001.patch
          590 kB
          Steve Loughran

        Issue Links

        depends upon

        Sub-task - The sub-task of the issue HADOOP-14488 s3guard w/ failure injection: listStatus fails after renaming file into directory

        • Blocker - Blocks development and/or testing work, production could not run
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14809 hadoop-aws shell profile not being built

        • Blocker - Blocks development and/or testing work, production could not run
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14215 DynamoDB client should waitForActive on existing tables

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14236 S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14288 TestDynamoDBMetadataStore is broken unless we can fail faster without a table version

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13589 S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13876 S3Guard: better support for multi-bucket access

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13914 s3guard: improve S3AFileStatus#isEmptyDirectory handling

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13926 S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13985 s3guard: add a version marker to every table

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13995 s3guard cli: make tests easier to run and address failure

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14013 S3Guard: fix multi-bucket integration tests

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14020 Optimize dirListingUnion

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14036 S3Guard: intermittent duplicate item keys failure

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14051 S3Guard: link docs from index, fix typos

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14448 Play nice with ITestS3AEncryptionSSEC

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14457 create() does not notify metadataStore of parent directories or ensure they're not existing files

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14484 Ensure deleted parent directory tombstones are overwritten when implicitly recreated

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14733 ITestS3GuardConcurrentOps failing with -Ddynamodblocal -Ds3guard

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14749 Review S3guard docs & code prior to merge

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14487 DirListingMetadata precondition failure messages to include path at fault

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14505 simplify mkdirs() after S3Guard delete tracking change

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14633 S3Guard: optimize create codepath

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14735 ITestS3AEncryptionSSEC failing in parallel s3guard runs

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14777 S3Guard premerge changes: java 7 build & test tuning

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        is depended upon by

        Sub-task - The sub-task of the issue HADOOP-14838 backport S3guard to branch-2

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14423 s3guard will set file length to -1 on a putObjectDirect(stream, -1) call

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stevel@apache.org Steve Loughran
            stevel@apache.org Steve Loughran
            Votes:
            1 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment