[FLINK-25491] Code generation: init method exceeds 64 KB when there is a large IN filter with Table API

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.2
• Fix Version/s: 1.15.0
• Component/s: Table SQL / API, Table SQL / Runtime
• Labels: pull-request-available
• Assignee: Caizhi Weng
• Reporter: Daniel Cheng
• Created: 31/Dec/21 01:56
• Updated: 21/Feb/22 02:32
• Resolved: 21/Feb/22 02:32

Description

When using Table API (Blink planner), if you are filtering using an IN filter with a lot of values, e.g. $(colName).in(<many values>), it will result in the error

 

Code of method "<init>(...)V" of class "BatchExecCal$3006" grows beyond 64 KB

 

The size of the init method mainly comes from the below method, which initializes the hash set with all the values in the filter.
addReusableHashSet in CodeGeneratorContext.scala

https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/CodeGeneratorContext.scala#L409

 

This affects older versions as well, with 1.14.2 being the latest version that exhibits this issue.

 

Issue Links

• links to: GitHub Pull Request #18675