Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-6445

Templating is not used on "parameters" field (but it should)

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.10.10
    • Fix Version/s: None
    • Component/s: database, operators
    • Labels:
      None

      Description

      In several database related classes (SqlSensor, MySqlOperator, MsSqlOperator, OracleOperator, PostgresOperator, JdbcOperator, SqliteOperator,  ...) exists a possibility to use parameters with a SQL statement, but templating is not enabled on the parameters, only on the SQL statement itself.

      Not using parameters has a performance impact on the database. The db has to parse a SQL statement to find an optimized way to execute it. Because this operation is a performance bottleneck the calculated execution plan is cached and reused when the exact same statement is executed again. But the database only caches a certain number of execution plans, if one application unnecessarily litters the cache with a lot of similar statements, other applications suffer. (More detailed explanation: https://www.akadia.com/services/ora_bind_variables.html)

      Therefore please add templating to the parameters so that the actual SQL statement doesn't need to change between runs.

      old:
      template_fields = ('sql',) # type: Iterable[str]

      new:
      template_fields = ('sql', 'parameters',) # type: Iterable[str]

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              moser.b Brian Moser
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: