Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7944 Improvements of portability framework to make it usable in other projects
  3. BEAM-7947

Improves the interfaces of classes such as FnDataService, BundleProcessor, ActiveBundle, etc to change the parameter type from WindowedValue<T> to T

Details

    • Sub-task
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.16.0
    • java-fn-execution
    • None

    Description

      Both `Coder<WindowedValue<T>>` and `FnDataReceiver<WindowedValue<T>>` use `WindowedValue` as the data structure that both sides of Runner and SDK Harness know each other. Control Plane/Data Plane/State Plane/Logging is a highly abstraction, such as Control Plane and Logging, these are common requirements for all multi-language platforms. For example, the Flink community is also discussing how to support Python UDF, as well as how to deal with docker environment. how to data transfer, how to state access, how to logging etc. If Beam can further abstract these service interfaces, i.e., interface definitions are compatible with multiple engines, and finally provided to other projects in the form of class libraries, it definitely will help other platforms that want to support multiple languages. Here I am to throw out a minnow to catch a whale, take the FnDataService#receive interface as an example, and turn `WindowedValue<T>` into `T` so that other platforms can be extended arbitrarily, as follows:

      <T> InboundDataClient receive(LogicalEndpoint inputLocation, Coder<T> coder, FnDataReceiver<T>> listener);
      

      Attachments

        Issue Links

          Activity

            People

              sunjincheng121 sunjincheng
              sunjincheng121 sunjincheng
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h