Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Invalid
-
None
-
None
-
None
-
None
-
Airflow 2.3.2
jinja 3.0.3 (also tried jinja 3.1.*)
Ubuntu 22.04, amd64, on AWS
Description
This seems odd and there may be another answer for it, but I can't find anything. We have been running 2.1.4 on Ubuntu Xenial for about a year without much trouble. So, we are switching over to the new Ubuntu 22.04 LTS environment with all up to date libraries.
When attempting to run Airflow 2.3.2, I am getting strange config-related errors when processing our custom config variable (full stack trace at the bottom).
- ` log_processor_filename_template = {{{{ filename }}}}.log` - This is the value included in the default template, but it fails to process with the error `jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'`
- The default `log_filename_template` value also fails to process with a similar error
Commenting these out allows the config file to process, however it is not substituting templates correctly. The multiple `{AIRFLOW_HOME}` references are being interpreted literally instead of being substituted for the actual value. If I change these to reference an environment variable (e.g., `${AIRFLOW_HOME}`), it works fine.
Here's the stack trace for the log filename template. I put in a custom breakpoint to determine that it indeed is just for the log_filename_template and log_processor_filename_template variables:
```
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Unable to load the config, contains a configuration error.
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 565, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: handler = self.configure_handler(handlers[name])
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 746, in configure_handler
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: result = factory(**kwargs)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/utils/log/file_processor_handler.py", line 44, in _init_
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: self.filename_template, self.filename_jinja_template = parse_template_string(filename_template)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/utils/helpers.py", line 176, in parse_template_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return None, jinja2.Template(template_string)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1195, in _new_
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return env.from_string(source, template_class=cls)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1092, in from_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return cls.from_code(self, self.compile(source), gs, None)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 757, in compile
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: self.handle_exception(source=source_hint)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise rewrite_traceback_stack(source=source)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "<unknown>", line 1, in template
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: The above exception was the direct cause of the following exception:
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/bin/airflow", line 5, in <module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: from airflow._main_ import main
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/_init_.py", line 47, in <module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: settings.initialize()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 561, in initialize
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: LOGGING_CLASS_PATH = configure_logging()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 73, in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise e
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 68, in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: dictConfig(logging_config)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 810, in dictConfig
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: dictConfigClass(config).configure()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 572, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise ValueError('Unable to configure handler '
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: ValueError: Unable to configure handler 'processor'
```