Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20491

Support Broadcast Operation in BATCH execution mode

    XMLWordPrintableJSON

    Details

      Description

      Right now, we don't support DataStream.connect(BroadcastStream) in BATCH execution mode. I believe we can add support for this with not too much work.

      The key insight is that we can process the broadcast side before the non-broadcast side. Initially, we were shying away from this because of concerns about ctx.applyToKeyedState() which allows the broadcast side of the user function to access/iterate over state from the keyed side. We thought that we couldn't support this. However, since we know that we process the broadcast side first we know that the keyed side will always be empty when doing so. We can thus just make this "keyed iteration" call a no-op, instead of throwing an exception as we do now.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                aljoscha Aljoscha Krettek
                Reporter:
                aljoscha Aljoscha Krettek
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: