[FLINK-7685] CompilerException: "Bug: Logic for branching plans (non-tree plans) has an error, and does not track the re-joining of branches correctly"

• Type: Bug
• Status: Reopened
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: API / DataSet
• Assignee: Unassigned
• Reporter: Gábor Gévay
• Created: 25/Sep/17 14:17
• Updated: 29/Jul/21 07:38

Description

A Flink program which reads an input DataSet, creates 64 new DataSets from it, and writes these to separate files throws the following exception:

Exception in thread "main" org.apache.flink.optimizer.CompilerException: Bug: Logic for branching plans (non-tree plans) has an error, and does not track the re-joining of branches correctly.
	at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:491)
	at org.apache.flink.optimizer.Optimizer.compile(Optimizer.java:398)
	at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:187)
	at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
	at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:921)
	at org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:86)

Here is some code that reproduces it:
https://github.com/ggevay/flink/tree/compiler-exception-new

Note that it works with less than 64 DataSets.

Also note that with more than 64 DataSets it throws CompilerException: Cannot currently handle nodes with more than 64 outputs, which is at least a clear error msg that helps the user to find a workaround.