Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.1.0
    • Fix Version/s: 3.1.2
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:

      Description

      I'm writing about the paper on the committers, one which, being a proper paper, requires me to show the committers work.

      1. define the requirements of a "Correct" committed job (this applies to the FileOutputCommitter too)
      2. show that the Staging committer meets these requirements (most of this is implicit in that it uses the V1 FileOutputCommitter to marshall .pendingset lists from committed tasks to the final destination, where they are read and committed.
      3. Show the magic committer also works.

        Attachments

        1. HADOOP-15107-001.patch
          35 kB
          Steve Loughran
        2. HADOOP-15107-002.patch
          44 kB
          Steve Loughran
        3. HADOOP-15107-003.patch
          42 kB
          Steve Loughran
        4. HADOOP-15107-004.patch
          44 kB
          Steve Loughran

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: