Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13345 S3Guard: Improved Consistency for S3A
  3. HADOOP-13998

Merge initial S3guard release into trunk

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Done
    • 3.0.0-beta1
    • 3.0.0-beta1
    • fs/s3
    • None

    Description

      JIRA to link in all the things we think are needed for a preview/merge into trunk

      Attachments

        1. HADOOP-13998-005.patch
          592 kB
          Steve Loughran
        2. HADOOP-13998-004.patch
          592 kB
          Steve Loughran
        3. HADOOP-13998-003.patch
          591 kB
          Steve Loughran
        4. HADOOP-13998-002.patch
          590 kB
          Steve Loughran
        5. HADOOP-13998-001.patch
          590 kB
          Steve Loughran

        Issue Links

        depends upon

        Sub-task - The sub-task of the issue HADOOP-14488 s3guard w/ failure injection: listStatus fails after renaming file into directory

        • Blocker - Blocks development and/or testing work, production could not run
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14809 hadoop-aws shell profile not being built

        • Blocker - Blocks development and/or testing work, production could not run
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14215 DynamoDB client should waitForActive on existing tables

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14236 S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14288 TestDynamoDBMetadataStore is broken unless we can fail faster without a table version

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13589 S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13876 S3Guard: better support for multi-bucket access

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13914 s3guard: improve S3AFileStatus#isEmptyDirectory handling

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13926 S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13985 s3guard: add a version marker to every table

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13995 s3guard cli: make tests easier to run and address failure

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14013 S3Guard: fix multi-bucket integration tests

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14020 Optimize dirListingUnion

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14036 S3Guard: intermittent duplicate item keys failure

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14051 S3Guard: link docs from index, fix typos

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14448 Play nice with ITestS3AEncryptionSSEC

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14457 create() does not notify metadataStore of parent directories or ensure they're not existing files

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14484 Ensure deleted parent directory tombstones are overwritten when implicitly recreated

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14733 ITestS3GuardConcurrentOps failing with -Ddynamodblocal -Ds3guard

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14749 Review S3guard docs & code prior to merge

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14487 DirListingMetadata precondition failure messages to include path at fault

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14505 simplify mkdirs() after S3Guard delete tracking change

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14633 S3Guard: optimize create codepath

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14735 ITestS3AEncryptionSSEC failing in parallel s3guard runs

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14777 S3Guard premerge changes: java 7 build & test tuning

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        is depended upon by

        Sub-task - The sub-task of the issue HADOOP-14838 backport S3guard to branch-2

        • Major - Major loss of function.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-14423 s3guard will set file length to -1 on a putObjectDirect(stream, -1) call

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stevel@apache.org Steve Loughran Assign to me
            stevel@apache.org Steve Loughran
            Votes:
            1 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment