Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-3539

Error Importing sub-dag (Webserver module load fail, scheduler works fine)

    XMLWordPrintableJSON

    Details

      Description

      Hi, I'm deploying the airflow in a VM instance in google cloud plataform, using the docker image of puckel/docker-airflow, and using the docker-compose-Celery-Executor.yaml. When I run the dags locally the imports seens fine. but when I deploy on this docker-compose the import seems wrong to the  webserver, the scheduler runs the dag fine, no problems at  sight, when I turn on the dag in the UI the dag runs in the scheduler/ worker. but on the webserver the import gives me this error:

      [2018-12-18 13:06:15,273] {{__init__.py:51}} INFO - Using executor CeleryExecutor
      [2018-12-18 13:06:15,762] {{models.py:271}} INFO - Filling up the DagBag from /usr/local/airflow/dags
      [2018-12-18 13:06:16,290] {{models.py:380}} ERROR - Failed to import: /usr/local/airflow/dags/dags/daily_etl.py
      Traceback (most recent call last):
       File "/usr/local/lib/python3.6/site-packages/airflow/models.py", line 377, in process_file
       m = imp.load_source(mod_name, filepath)
       File "/usr/local/lib/python3.6/imp.py", line 172, in load_source
       module = _load(spec)
       File "<frozen importlib._bootstrap>", line 684, in _load
       File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
       File "<frozen importlib._bootstrap_external>", line 678, in exec_module
       File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
       File "/usr/local/airflow/dags/dags/daily_etl.py", line 4, in <module>
       from dags.daily_extract import dag as dag_extract
      ModuleNotFoundError: No module named 'dags.daily_extract'
      

       

        Attachments

        1. Seleção_011.png
          16 kB
          Leonardo Zanell

          Activity

            People

            • Assignee:
               alexlumpov Alex Lumpov
              Reporter:
              lvcz Leonardo Zanell
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: