Details
-
Sub-task
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
Both `Coder<WindowedValue<T>>` and `FnDataReceiver<WindowedValue<T>>` use `WindowedValue` as the data structure that both sides of Runner and SDK Harness know each other. Control Plane/Data Plane/State Plane/Logging is a highly abstraction, such as Control Plane and Logging, these are common requirements for all multi-language platforms. For example, the Flink community is also discussing how to support Python UDF, as well as how to deal with docker environment. how to data transfer, how to state access, how to logging etc. If Beam can further abstract these service interfaces, i.e., interface definitions are compatible with multiple engines, and finally provided to other projects in the form of class libraries, it definitely will help other platforms that want to support multiple languages. Here I am to throw out a minnow to catch a whale, take the FnDataService#receive interface as an example, and turn `WindowedValue<T>` into `T` so that other platforms can be extended arbitrarily, as follows:
<T> InboundDataClient receive(LogicalEndpoint inputLocation, Coder<T> coder, FnDataReceiver<T>> listener);
Attachments
Issue Links
- links to