[FLINK-23600] Rework StateFun's remote module parsing and binding

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: statefun-3.1.0
• Component/s: Stateful Functions
• Labels: pull-request-available
• Assignee: Tzu-Li (Gordon) Tai
• Reporter: Tzu-Li (Gordon) Tai
• Created: 03/Aug/21 08:48
• Updated: 27/Aug/21 15:33
• Resolved: 09/Aug/21 11:24

Description

Currently, we have a JsonModule class that is responsible for parsing user's module YAML specifications, resolving the specification into application components (i.e. function providers, ingresses, routers, and egresses) that is then bound to the application universe.

Over time, the JsonModule class has overgrown with several changes as we progressively adapted the YAML format.

• The class handles ALL kinds of components, including ingresses / functions / egresses etc. The code is extremely fragile and becoming hard to extend.
• Users have no access to extend this class, if they somehow need to plugin custom components (e.g. adding an unsupported ingress / egress, custom protocol implementations etc).

We aim to rework this with the following goals in mind:

1. The system should only handle module.yaml parsing up to the point where it extracts a list of JSON objects that individually represent an application component.
2. The system has no knowledge of what each JSON objects contains, other than its TypeName which would map to a corresponding ComponentBinder.
3. A ComponentBinder is essentially an extension bound to the system that knows how to parse a specific JSON object, and bind components to the application universe.

Issue Links

• links to: GitHub Pull Request #247
• links to: GitHub Pull Request #263