Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-2739

Airflow crashes on startup if LC_ALL env isnt set to utf-8

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.10.0
    • Component/s: configuration
    • Labels:
      None
    • Environment:
      Python 3.6.0, Ubuntu 14.04.5 LTS

      Description

      When running Airflow 1.10.0 RC1 without LC_ALL environment variable set Airflow crashes on start with the following trace

       

      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 Traceback (most recent call last):
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01   File "/opt/virtualenv/tictail/pipeline/bin/airflow", line 21, in <module>
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01     from airflow import configuration
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01   File "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-packages/airflow/__init__.py", line 35, in <module>
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01     from airflow import configuration as conf
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01   File "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-packages/airflow/configuration.py", line 106, in <module>
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01     DEFAULT_CONFIG = f.read()
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01   File "/opt/virtualenv/tictail/pipeline/lib/python3.6/encodings/ascii.py", line 26, in decode
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01     return codecs.ascii_decode(input, self.errors)[0]
      Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 20770: ordinal not in range(128)
      

      This is because the `config_templates/default_airflow.cfg` contains a non-ascii character and if LC_ALL isnt set to `en_US.UTF-8` or similar Python will assume the file is in ascii.

       

      Solution would be to always open the config files as utf-8 regardless of the LC_ALL environment variable.

       

      This worked up until https://github.com/apache/incubator-airflow/commit/16bae5634df24132b37eb752fe816f51bf7e83ca it seems.

       

      Python versions affected, 3.4.0, 3.5.5, 3.6.0

        Attachments

          Activity

            People

            • Assignee:
              cjg Carl Johan Gustavsson
              Reporter:
              cjg Carl Johan Gustavsson
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: