Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-323

Should be able to prevent tasks from overlapping across multiple DAG Runs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Auto Closed
    • Affects Version/s: 1.7.1.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:
      1.7.1.2

      Description

      As a the Airflow administrator,
      If a task from a previous DAG Run is still running when the next scheduled run triggers the same task, there should be a way prevent the tasks from overlapping.
      Otherwise the same code could end up running multiple times simultaneously.

      To reproduce:
      1) Create a DAG with a short scheduled interval
      2) Create a task in that DAG to run longer than the interval
      Result: Both tasks end up running that the same time.

      This can cause tasks to compete for resources as well as duplicating or overwriting what the other task is doing.

        Attachments

          Activity

            People

            • Assignee:
              isteele Isaac Steele
              Reporter:
              isteele Isaac Steele
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: