[KAFKA-10307] Topology cycles in KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 2.4.0, 2.5.0, 2.6.0
• Component/s: streams
• Assignee: Unassigned
• Reporter: Boyang Chen
• Created: 25/Jul/20 06:02
• Updated: 28/Aug/20 14:28
• Resolved: 28/Aug/20 14:27

Description

We have spotted a cycled topology for the foreign-key join test shouldInnerJoinMultiPartitionQueryable, not sure yet whether this is a bug in the algorithm or the test only. Used https://zz85.github.io/kafka-streams-viz/ to visualize:

Sub-topology: 0
    Source: KTABLE-SOURCE-0000000019 (topics: [KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-0000000017-topic])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-RESOLVER-PROCESSOR-0000000020
    Source: KTABLE-SOURCE-0000000032 (topics: [KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-0000000030-topic])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-RESOLVER-PROCESSOR-0000000033
    Source: KSTREAM-SOURCE-0000000001 (topics: [table1])
      --> KTABLE-SOURCE-0000000002
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-RESOLVER-PROCESSOR-0000000020 (stores: [table1-STATE-STORE-0000000000])
      --> KTABLE-FK-JOIN-OUTPUT-0000000021
      <-- KTABLE-SOURCE-0000000019
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-RESOLVER-PROCESSOR-0000000033 (stores: [INNER-store1])
      --> KTABLE-FK-JOIN-OUTPUT-0000000034
      <-- KTABLE-SOURCE-0000000032
    Processor: KTABLE-FK-JOIN-OUTPUT-0000000021 (stores: [INNER-store1])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000023
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-RESOLVER-PROCESSOR-0000000020
    Processor: KTABLE-FK-JOIN-OUTPUT-0000000034 (stores: [INNER-store2])
      --> KTABLE-TOSTREAM-0000000035
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-RESOLVER-PROCESSOR-0000000033
    Processor: KTABLE-SOURCE-0000000002 (stores: [table1-STATE-STORE-0000000000])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000010
      <-- KSTREAM-SOURCE-0000000001
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000010 (stores: [])
      --> KTABLE-SINK-0000000011
      <-- KTABLE-SOURCE-0000000002
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000023 (stores: [])
      --> KTABLE-SINK-0000000024
      <-- KTABLE-FK-JOIN-OUTPUT-0000000021
    Processor: KTABLE-TOSTREAM-0000000035 (stores: [])
      --> KSTREAM-SINK-0000000036
      <-- KTABLE-FK-JOIN-OUTPUT-0000000034
    Sink: KSTREAM-SINK-0000000036 (topic: output-)
      <-- KTABLE-TOSTREAM-0000000035
    Sink: KTABLE-SINK-0000000011 (topic: KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000009-topic)
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000010
    Sink: KTABLE-SINK-0000000024 (topic: KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000022-topic)
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000023  Sub-topology: 1
    Source: KSTREAM-SOURCE-0000000004 (topics: [table2])
      --> KTABLE-SOURCE-0000000005
    Source: KTABLE-SOURCE-0000000012 (topics: [KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000009-topic])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000014
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000014 (stores: [KTABLE-FK-JOIN-SUBSCRIPTION-STATE-STORE-0000000013])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000015
      <-- KTABLE-SOURCE-0000000012
    Processor: KTABLE-SOURCE-0000000005 (stores: [table2-STATE-STORE-0000000003])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000016
      <-- KSTREAM-SOURCE-0000000004
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000015 (stores: [table2-STATE-STORE-0000000003])
      --> KTABLE-SINK-0000000018
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000014
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000016 (stores: [KTABLE-FK-JOIN-SUBSCRIPTION-STATE-STORE-0000000013])
      --> KTABLE-SINK-0000000018
      <-- KTABLE-SOURCE-0000000005
    Sink: KTABLE-SINK-0000000018 (topic: KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-0000000017-topic)
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000015, KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000016  Sub-topology: 2
    Source: KSTREAM-SOURCE-0000000007 (topics: [table3])
      --> KTABLE-SOURCE-0000000008
    Source: KTABLE-SOURCE-0000000025 (topics: [KTABLE-FK-JOIN-SUBSCRIPTION-REGISTRATION-0000000022-topic])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000027
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000027 (stores: [KTABLE-FK-JOIN-SUBSCRIPTION-STATE-STORE-0000000026])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000028
      <-- KTABLE-SOURCE-0000000025
    Processor: KTABLE-SOURCE-0000000008 (stores: [table3-STATE-STORE-0000000006])
      --> KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000029
      <-- KSTREAM-SOURCE-0000000007
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000028 (stores: [table3-STATE-STORE-0000000006])
      --> KTABLE-SINK-0000000031
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000027
    Processor: KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000029 (stores: [KTABLE-FK-JOIN-SUBSCRIPTION-STATE-STORE-0000000026])
      --> KTABLE-SINK-0000000031
      <-- KTABLE-SOURCE-0000000008
    Sink: KTABLE-SINK-0000000031 (topic: KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-0000000030-topic)
      <-- KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000028, KTABLE-FK-JOIN-SUBSCRIPTION-PROCESSOR-0000000029