[KAFKA-9020] Streams sub-topologies should be sorted by sink -> source relationship

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.4.0
• Component/s: streams
• Assignee: A. Sophie Blee-Goldman
• Reporter: Guozhang Wang
• Created: 10/Oct/19 21:55
• Updated: 14/Oct/19 20:28
• Resolved: 14/Oct/19 20:19

Description

In Streams InternalTopologyBuilder we are sorting the sub-topologies (named as node groups) based on the source node names:

for (final String nodeName : Utils.sorted(allSourceNodes)) {
            nodeGroupId = putNodeGroupName(nodeName, nodeGroupId, nodeGroups, rootToNodeGroup);
        }

This is okay for DSL where the source node names are not customized, and hence we can just rely on the internal suffix ..-SOURCE-xxxxx which is basically following the parsing order. But now with the source node name customized with KIP-307, we should no longer rely on the source node names. Instead, we should sort sub-topologies by the sink -> source relationship — that is, if a sub-topology A has a sink node sending to topic X while another sub-topology B has a source node reading from X, then A should be sorted before B.

Issue Links

• links to: GitHub Pull Request #7495