[FLINK-23007] Introduce java code splitter to split generated code

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 1.14.0
• Component/s: Table SQL / Runtime
• Assignee: Caizhi Weng
• Reporter: Jingsong Lee
• Created: 16/Jun/21 07:25
• Updated: 14/Apr/22 10:00
• Resolved: 26/Jul/21 09:21

Description

Current lots of "code of method of class grows beyond 64k".

We have fixed by:

https://issues.apache.org/jira/browse/FLINK-15430

https://issues.apache.org/jira/browse/FLINK-16589

But this approach is not perfect. It has many corner cases.

 

We can introduce a post java class splitter, based on antlr4, anaysis code and split method to multi-methods.

 

We can add this splitter in CompileUtils, then, every compile class can be split.

Issue Links

• is related to: FLINK-16589 Flink Table SQL fails/crashes with big queries with lots of fields
• is related to: FLINK-22788 Code of equals method grows beyond 64 KB
• is related to: FLINK-22903 Code of method xxx of class "StreamExecCalc$1248" grows beyond 64 KB
• is related to: FLINK-27246 Code of method "processElement(Lorg/apache/flink/streaming/runtime/streamrecord/StreamRecord;)V" of class "HashAggregateWithKeys$9211" grows beyond 64 KB
• is related to: FLINK-20898 Code of BatchExpand & LocalNoGroupingAggregateWithoutKeys grows beyond 64 KB
• is related to: FLINK-15430 Fix Java 64K method compiling limitation for blink planner.
• is related to: FLINK-19363 Code of split method grows beyond 64 KB

Sub-Tasks

• Introduce Java code splitter for code generation: Closed
• flink-table-code-splitter: NOTICE should in META-INF: Closed
• Activate Java code splitter for all generated classes: Closed
• Clean up the old implementation of code splitting: Closed
• Clean up logs produced by code splitter: Closed
• JavaCodeSplitter can not split the code from ProjectionCodeGenerator: Closed
• Check all GeneratedClass to make sure they can be split by JavaCodeSplitter: Closed