Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-6769

Multiple DAGs cause infinite loop in logging

Agile BoardAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersLinkUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.10.9
    • Fix Version/s: None
    • Component/s: scheduler
    • Labels:
    • Environment:
      macOS 10.15.2
      Python 3.7.5


      After adding a second DAG to Airflow the scheduler spirals into an infinite loop and is unable to schedule any tasks. The recursion appears to have to do with logging. I've attached the output from scheduler and my requirements.txt

      If I reset Airflow (`airflow resetdb`) and remove all but 1 DAG from the folder there is no issue. I don't believe it's a specific DAG as I have tried with various different test ones. I don't even have to schedule/run them, just having Airflow recognize them causes the loop. 

      I'm using MySQL and Celery as my store and worker. 

      I have also attached part of the log file for one of my DAGs outputted by the scheduler ($AIRFLOW_HOME/logs/scheduler/{dag_name}/latest/{dag_name}.log). It seems `collect_dags` is being called every second by the scheduler. 


        1. scheduler-dag.log
          4 kB
          Chris Boden
        2. requirements.txt
          3 kB
          Chris Boden
        3. scheduler.log
          6 kB
          Chris Boden




              • Created:

                Issue deployment