Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-6988

Task queue is not reloaded on TI or DagRun clear

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.10.9
    • Fix Version/s: None
    • Component/s: scheduler
    • Labels:
      None

      Description

      Hello,

      I am not sure if this is a bug or a feature:

      If I update the queue configuration in a DAG, Airflow persists the old queue names on clearing the existing tasks or DAG runs. Scheduling new tasks on the same DAG however take the new configuration. I'd prefer not to rename the DAG and start a fresh as the tasks are long and costly to execute.

      This is problematic as your infrastructure has to then support legacy queues.

      I will be providing a patch to fix the issue in 1.10 but may need some guidance with 2.0 as I believe the scheduler internals have changed a lot.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              robinedwards Robin Edwards
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: