Uploaded image for project: 'Apache Apex Core'
  1. Apache Apex Core
  2. APEXCORE-235 Improve support for batch processing
  3. APEXCORE-408

Ability to schedule Sub-DAG from running application

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Today it is possible to add operators to a running application from the CLI/client. It should be possible to do this from within the application also. This can be used to expand/remove a Sub-DAG on demand, triggered by application specific logic. It will enable use cases such as batch applications that perform multiple stages of processing where not all resources are required at the same time or the logic of subsequent steps depends on the execution of previous steps.

      Attachments

        Activity

          People

            tushargosavi Tushar Gosavi
            thw Thomas Weise
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: