Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-5066

Usage of FieldRef in Kubernetes env execution

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.10.4, 2.0.0, 1.10.5
    • Fix Version/s: 1.10.7
    • Component/s: executors
    • Labels:
      None
    • Flags:
      Patch

      Description

      In Kubernetes, dynamic pod metadata can be set as environment variables and then used as substitution in other environment variables.

      Currently in Airflow, the pod request creation set those variables after the ones that can use substitutions so them doen't do it.

      Example usage is:

      • define a variable SPARK_LOCAL_IP using the pod metadata status.podIP
      • using this variable in Spark options as -Dspark.driver.host=$(SPARK_LOCAL_IP)

      I will do a PR to fix this behavior.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jsurloppe Julien Surloppe
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: