Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-2338

Airflow requires Hive config

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 1.10.0
    • Fix Version/s: None
    • Component/s: configuration
    • Labels:
      None

      Description

      Had working airflow, upgraded to latest from master branch, started getting config exceptions:

      [2018-04-16 23:52:11,732] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 [2018-04-16 23:52:11,731] {__init__.py:50} INFO - Using executor CeleryExecutor
      [2018-04-16 23:52:12,478] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 Traceback (most recent call last):
      [2018-04-16 23:52:12,478] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File "/var/lib/celery/venv/bin/airflow", line 32, in <module>
      [2018-04-16 23:52:12,478] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 args.func(args)
      [2018-04-16 23:52:12,478] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File "/var/lib/celery/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 77, in wrapper
      [2018-04-16 23:52:12,478] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 raise e
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File "/var/lib/celery/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 74, in wrapper
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 return f(*args, **kwargs)
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File "/var/lib/celery/venv/lib/python3.6/site-packages/airflow/bin/cli.py", line 438, in run
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 conf.set(section, option, value)
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File "/var/lib/celery/venv/lib/python3.6/site-packages/backports/configparser/__init__.py", line 1239, in set
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 super(ConfigParser, self).set(section, option, value)
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 File "/var/lib/celery/venv/lib/python3.6/site-packages/backports/configparser/__init__.py", line 921, in set
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 raise from_none(NoSectionError(section))
      [2018-04-16 23:52:12,479] {base_task_runner.py:106} INFO - Job 147: Subtask login_task0 backports.configparser.NoSectionError: No section: 'hive'
      [2018-04-16 23:52:36,727] {logging_mixin.py:95} INFO - [2018-04-16 23:52:36,726] \{jobs.py:2548} INFO - Task exited with return code 1
      

      It's looking for a hive section in the airflow.cfg but my config doesn't have one.

      Not sure what changed to make this break, but the default values should be used / we should not error out on a missing section that isn't even needed or used by current dags.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                johnarnold John Arnold
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: