Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-4582

Set KubernetesExecutor worker pod limits and requests based on "resources" task attribute

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.10.3
    • Fix Version/s: None
    • Component/s: executors

      Description

      At this time the KubernetesExecutor does not honor resource limits or requests set in the resources task attribute.

      It is possible to set limits and requests by passing an executor_config dict, however this method seems sub-optimal, considering that a generic built-in task attribute with essentially the same purpose is already available.

      I think we should implement this functionality, but gate it behind an opt-in configuration parameter to ensure backwards-compatibility.

        Attachments

          Activity

            People

            • Assignee:
              pgag Philippe Gagnon
              Reporter:
              pgag Philippe Gagnon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: