Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-690 Tez API Ease of Use
  3. TEZ-694

Remove task commit burden from user code

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Its easy to forget to add commit logic in the processor after the logic in the run() method is done. After that its hard to debug why the outputs are not appearing after the dag completes even though all the tasks have been completed successfully. Since commit is an operation that should happen after the processor completes, it cannot be delegated entirely to the outputs. We can either do it in the LIORuntimeTask after processor.run() completes or create and abstract base processor class that does this after the real run() method completes.

        Attachments

          Activity

            People

            • Assignee:
              kamrul Mohammad Islam
              Reporter:
              bikassaha Bikas Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: