Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-401

scheduler gets stuck without a trace

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.7.1.3
    • Fix Version/s: None
    • Component/s: executors, scheduler
    • Labels:

      Description

      The scheduler gets stuck without a trace or error. When this happens, the CPU usage of scheduler service is at 100%. No jobs get submitted and everything comes to a halt. Looks it goes into some kind of infinite loop.
      The only way I could make it run again is by manually restarting the scheduler service. But again, after running some tasks it gets stuck. I've tried with both Celery and Local executors but same issue occurs. I am using the -n 3 parameter while starting scheduler.

      Scheduler configs,
      job_heartbeat_sec = 5
      scheduler_heartbeat_sec = 5
      executor = LocalExecutor
      parallelism = 32

      Please help. I would be happy to provide any other information needed

        Attachments

        1. scheduler_stuck_7hours.png
          8 kB
          Nadeem Ahmed Nazeer
        2. Dag_code.txt
          5 kB
          Nadeem Ahmed Nazeer
        3. scheduler_stuck.png
          217 kB
          Nadeem Ahmed Nazeer
        4. schduler_cpu100%.png
          8 kB
          Nadeem Ahmed Nazeer

          Activity

            People

            • Assignee:
              ash Ash Berlin-Taylor
              Reporter:
              nadeem Nadeem Ahmed Nazeer
            • Votes:
              6 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

              • Created:
                Updated: