[FLINK-15674] Let Java and Scala Type Extraction go through the same stack

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: API / DataStream
• Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned usability
• Assignee: Unassigned
• Reporter: Stephan Ewen
• Created: 19/Jan/20 23:20
• Updated: 17/Apr/22 22:38

Description

Currently, the Java and Scala Type Extraction stacks are completely different.

• Java uses the TypeExtractor
• Scala uses the type extraction macros.

As a result, the same class can be extracted as different types in the different stacks, which can lead to very confusing results. In particular, when you use the TypeExtractor on Scala Classes, you always get a GenericType.

Suggestion for New Design

There should be one type extraction stack, based on the TypeExtractor.

• The TypeExtractor should be extensible and load additions through service loaders, similar as it currently loads Avro as an extension.
• The Scala Type Extraction logic should be such an extension.
• The Scala Marcos would only capture the Type (as in Java type), meaning Class, or ParameterizedType, or Array (etc.) and delegate this to the TypeExtractor.

Sub-Tasks

• Decouple resolving Type from creating TypeInformation process: In Progress
• Introduce TypeInformationExtractor famework: Open
• Migrate current TypeInformation creation to the TypeInformationExtractor framework: Open