[FLINK-17171] Blink planner fails to compile Table program with POJO source

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 1.10.0
• Component/s: Table SQL / Planner, Table SQL / Runtime
• Assignee: Unassigned
• Reporter: Nico Kruber
• Created: 15/Apr/20 17:08
• Updated: 16/Apr/20 05:47
• Resolved: 16/Apr/20 05:47

Description

It seems as if FLINK-13993 made the Table API (Blink planner) unusable for POJO sources where the POJO class is in user code.

For https://github.com/ververica/lab-sql-vs-datastream/blob/master/src/main/java/com/ververica/LateralTableJoin.java I get the following Exception when I run it on a Flink 1.10.0 cluster (full version attached):

2020-04-15 17:19:15,561 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Unhandled exception.
org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
...
Caused by: org.codehaus.commons.compiler.CompileException: Line 28, Column 175: Cannot determine simple type name "com"
...
	at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78)

I enabled debug logs and this is what it is trying to compile:

        @Override
        public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord element) throws Exception {
          org.apache.flink.table.dataformat.BaseRow in1 = (org.apache.flink.table.dataformat.BaseRow) (org.apache.flink.table.dataformat.BaseRow) converter$15.toInternal((com.ververica.tables.FactTable.Fact) element.getValue());
...

I use a standalone cluster and submit via web UI and also verified that my jar file does not contain anything else but its compiled classes.

This code is working fine inside the IDE and was also working with Flink 1.10 and VVP 2.0 which did not use a dedicated class loader for user code.

My guess is that the (generated) code does not have access to FactTable.Fact and the Janino compiler does not produce the right error message seeing "com" as a primitive type instead.

FLINK-7490 and FLINK-9220 seem related but too old (legacy planner).

Issue Links

• duplicates: FLINK-16662 Blink Planner failed to generate JobGraph for POJO DataStream converting to Table (Cannot determine simple type name)