Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21036

Consider removing automatic configuration fo number of slots from docker

    XMLWordPrintableJSON

    Details

    • Release Note:
      The docker images no longer set the default number of taskmanager slots to the number of CPU cores. This behavior was inconsistent with all other deployment methods and ignored any limits on the CPU usage set via docker.

      Description

      The docker-entrypoint.sh supports setting the number of task slots via the TASK_MANAGER_NUMBER_OF_TASK_SLOTS environment variable, which defaults to the number of cpu cores via $(grep -c ^processor /proc/cpuinfo).

      The environment variable itself is redundant nowadays since we introduced FLINK_PROPERTIES, and is no longer documented.

      Defaulting to the number of CPU cores can be considered convenience, but it seems odd to have this specific to docker while the distribution defaults to 1.
      The bigger issue in my mind though is that this creates a configuration mismatch between the Job- and TaskManager processes; the ResourceManager specifically needs to know how many slots a worker has to make decisions about redundancy and allocating resources.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chesnay Chesnay Schepler
                Reporter:
                chesnay Chesnay Schepler
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: