Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-3047

Local batch execution: set number of task manager slots to the maximum parallelism

    XMLWordPrintableJSON

    Details

      Description

      The number of task slots for local execution are determined by the maximum parallelism found. However, if a default parallelism has been set, this parallelism is used as the upper bound for the number of task slots.

      We should change this to always use the maximum parallelism as the number of task slots. Otherwise jobs which include operators with a parallelism higher than the default parallelism fail to execute locally.

      For example, this fails

      ExecutionEnvironment env = ..
      
      env.setParallelism(2);
      
      DataSet<Integer> set = env.fromElements(1,2,3,4)
          .map(el -> el+1)
          .setParallelism(4);
      
      set.print();
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mxm Maximilian Michels
                Reporter:
                mxm Maximilian Michels
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: