[FLINK-13191] Add a simplified UDF type extraction

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Table SQL / API
• Assignee: Timo Walther
• Reporter: Timo Walther
• Created: 10/Jul/19 09:42
• Updated: 13/Apr/21 05:46

Description

Currently, the input, accumulator, and output types of UDFs are partially defined within an UDF and outside of the UDF in language-specific table environments (Scala vs. Java). This makes UDFs environment-dependent and not modular. It also blocks the unified org.apache.flink.table.api.TableEnvironment.

A better approach is to perform the type extraction Table API specific within the UDF and let users add additional information through UDF annotations.

An exact annotation and extraction design will follow.

Issue Links

• causes: FLINK-20747 ClassCastException when using MAX aggregate function
• relates to: FLINK-16268 Failed to run rank over window with Hive built-in functions
• mentioned in: Page Loading...
• mentioned in: Page Loading...

Sub-Tasks

• Add a utility for creating data types via reflection: Closed
• Add interfaces for input type inference: Closed
• Merge InputTypeStrategy and InputTypeValidator: Closed
• Add a utility for creating type inference logic via reflection: Closed
• Map Flink's TypeInference to Calcite's interfaces: Closed
• Implement a DataTypeLookup: Closed
• Expose the new type inference for scalar functions: Closed
• Expose the new type inference for table functions: Closed
• Avoid unnecessary casting in TypeInferenceOperandChecker: Resolved
• Support new type inference for temporal table functions: Closed
• Ensure deterministic type inference extraction: Closed
• Use new type inference for SQL table and scalar functions: Closed
• Add documentation for new scalar/table functions: Closed
• Scala varargs cause exception for new inference: Closed
• Update UNNEST to new type system: Closed
• Invalid error message for overloaded methods with same parameter name: Closed
• Expose the new type inference for aggregate functions: Closed
• Update internal aggregate functions to new type system: Closed
• Use new type inference for SQL DDL of aggregate functions: Closed
• Update documentation about user-defined aggregate functions: Closed
• Map views are deserialized multiple times: Closed
• Support the new type inference in Scala Table API scalar functions: Closed
• Support the new type inference in Scala Table API table functions: Closed
• Support the new type inference in Scala Table API aggregate functions: Open
• Update test utilities to FLIP-65: Open
• Use the new type inference for hive udf: Closed
• Use the new type inference for hive udtf: Closed
• Use the new type inference for hive udaf: Closed
• Support new type inference for Table#flatMap: Closed
• Support implicit conversions of rows: Closed
• Support @DataTypeHint for TableFunction output types: Closed
• Update LookupTableSource to the new type system: Closed
• Make implementing a built-in function straightforward: Closed
• Add possibility to call built-in functions in SpecializedFunction: Closed
• Overloaded aggregate functions cause converter errors: Closed
• Drop old user-defined function stack: Open
• Use upper case name for all BuiltInFunctionDefinitions: Open
• TableEnvironment.createTemporarySystemFunction cause NPE when using leftOuterLateralJoin: Open
• Update max aggregate functions to new type system: Open