Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16829 Über-jira: S3A Hadoop 3.3.1 features
  3. HADOOP-17066

S3A staging committer committing duplicate files

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 3.3.0, 3.2.1, 3.1.3
    • None
    • fs/s3
    • None

    Description

      SPARK-31911 reporting concurrent jobs double writing files.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. SPARK-31911 Using S3A staging committer, pending uploads are committed more than once and listed incorrectly in _SUCCESS data

          • Major - Major loss of function.
          • Resolved
          Is contained by

          Sub-task - The sub-task of the issue HADOOP-17318 S3A committer to support concurrent jobs with same app attempt ID & dest dir

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: