Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4446

Support for vertex level commit for parallel outputs in Tez

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      By default, Tez does AM level commit controlled by the setting TEZ_AM_COMMIT_ALL_OUTPUTS_ON_DAG_SUCCESS. It has support for vertex level commit as well and that makes sense in some cases and Pig should support it.

      Attachments

        Issue Links

          is part of

          Bug - A problem which impairs or prevents the functions of the product. PIG-4487 Pig on Tez gives wrong success message on failure in case of multiple outputs

          • Major - Major loss of function.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. TEZ-714 OutputCommitters should not run in the main AM dispatcher thread

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              Unassigned Unassigned
              rohini Rohini Palaniswamy
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: