Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21421

Add coMapWithState to ConnectedStreams

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Minor
    • Resolution: Won't Do
    • None
    • None
    • API / Scala
    • None

    Description

      Currently there is no syntactic sugar for stateful functions in `ConnectedStreams` in Scala. 

      This makes stateful joins (aka `connect`) more verbose and exposes users to Java interfaces (by requiring a `RichCoMapFunction` implementation to access state in `ConnectedStreams`).

      Looking at DriveTribe's codebase, we have implemented ~80% of our ConnectedStreams operators using this `coMapWithState` implementation:

      https://github.com/ariskk/flink-stream-join/blob/main/src/main/scala/com/ariskk/streamjoin/ConnectedStreamsOps.scala#L15

      A `coFlatMapWithState` can be trivially implemented on top.

      This has been in production for so long I forgot it was our code and not Flink's.

      I can easily add it if this is of interest. 

      I did the work, here is the diff https://github.com/apache/flink/compare/master...ariskk:FLINK-21421

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            ariskoliopoulos Aris Koliopoulos
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: