Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-2873

Improvements to Quick Start flow

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.9.0
    • Fix Version/s: 1.10.0
    • Component/s: configuration
    • Labels:
      None

      Description

      Thank you for developing Airflow!

      Having ran through the Quick Start, i've come across two issues that I would like to highlight:

      bash-3.2$ cd ~/project/airflow/
      bash-3.2$ export AIRFLOW_HOME=~/project/airflow
      bash-3.2$ python3 -m venv $AIRFLOW_HOME/venv
      bash-3.2$ source venv/bin/activate
      (venv) bash-3.2$ pip install --upgrade pip
      (venv) bash-3.2$ uname -a Darwin mac.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
      (venv) bash-3.2$ python -V
      Python 3.6.5
      (venv) bash-3.2$ pip -V
      pip 18.0 from ~/project/airflow/venv/lib/python3.6/site-packages/pip (python 3.6)
      (venv) bash-3.2$ pip install apache-airflow[redis,postgres] -U 

      Results in:

      During handling of the above exception, another exception occurred:Traceback (most recent call last):
      File "~/project/airflow/venv/lib/python3.6/site-packages/airflow/models.py", line 639, in set_extra
      fernet = get_fernet()
      File "~/project/airflow/venv/lib/python3.6/site-packages/airflow/models.py", line 103, in get_fernet
      raise AirflowException('Failed to import Fernet, it may not be installed')
      airflow.exceptions.AirflowException: Failed to import Fernet, it may not be installed
      [2018-08-08 10:49:01,121]{models.py:643}ERROR - Failed to load fernet while encrypting value, using non-encrypted value.
      Traceback (most recent call last):
      File "~/project/airflow/venv/lib/python3.6/site-packages/airflow/models.py", line 101, in get_fernet
      from cryptography.fernet import Fernet
      ModuleNotFoundError: No module named 'cryptography'

      This is solved by:

      (venv) bash-3.2$ pip install cryptography

      Proposed fix:

      Include the `cryptography` package in the setup / package requirements

       

      Having fixed that, the following issue occurs when trying to:

      (venv) bash-3.2$ airflow initdb

      Exempt:

      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
      File "~/project/airflow/venv/lib/python3.6/site-packages/airflow/models.py", line 639, in set_extra
      fernet = get_fernet()
      File "~/project/airflow/venv/lib/python3.6/site-packages/airflow/models.py", line 107, in get_fernet
      raise AirflowException("Could not create Fernet object: {}".format(ve))
      airflow.exceptions.AirflowException: Could not create Fernet object: Incorrect padding
      [2018-08-08 10:50:50,697]
      
      {models.py:643}
      
      ERROR - Failed to load fernet while encrypting value, using non-encrypted value.
      Traceback (most recent call last):
      File "~/project/airflow/venv/lib/python3.6/site-packages/airflow/models.py", line 105, in get_fernet
      return Fernet(configuration.get('core', 'FERNET_KEY').encode('utf-8'))
      File "~/project/airflow/venv/lib/python3.6/site-packages/cryptography/fernet.py", line 34, in _init_
      key = base64.urlsafe_b64decode(key)
      File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/base64.py", line 133, in urlsafe_b64decode
      return b64decode(s)
      File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/base64.py", line 87, in b64decode
      return binascii.a2b_base64(s)
      binascii.Error: Incorrect padding

      Which after some googling leads to the conclusion that the ~/project/airflow/airflow.cfg fernet_key field is not set to the correct value.

      Feature request:

      Have the setup automatically generate a valid fernet key for the user.

      The fact that this page exists: https://bcb.github.io/airflow/fernet-key suggests this could easily be a part of the package.

      I understand that this project is in incubator phase, but I would say having a quick start that is not working as-is will discourage users from trying out this project. Thank you for considering.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ggeijteman G. Geijteman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: