Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-17171

Blink planner fails to compile Table program with POJO source

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      It seems as if FLINK-13993 made the Table API (Blink planner) unusable for POJO sources where the POJO class is in user code.

      For https://github.com/ververica/lab-sql-vs-datastream/blob/master/src/main/java/com/ververica/LateralTableJoin.java I get the following Exception when I run it on a Flink 1.10.0 cluster (full version attached):

      2020-04-15 17:19:15,561 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Unhandled exception.
      org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
      ...
      Caused by: org.codehaus.commons.compiler.CompileException: Line 28, Column 175: Cannot determine simple type name "com"
      ...
      	at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78)
      

      I enabled debug logs and this is what it is trying to compile:

              @Override
              public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord element) throws Exception {
                org.apache.flink.table.dataformat.BaseRow in1 = (org.apache.flink.table.dataformat.BaseRow) (org.apache.flink.table.dataformat.BaseRow) converter$15.toInternal((com.ververica.tables.FactTable.Fact) element.getValue());
      ...
      

      I use a standalone cluster and submit via web UI and also verified that my jar file does not contain anything else but its compiled classes.

      This code is working fine inside the IDE and was also working with Flink 1.10 and VVP 2.0 which did not use a dedicated class loader for user code.

      My guess is that the (generated) code does not have access to FactTable.Fact and the Janino compiler does not produce the right error message seeing "com" as a primitive type instead.

      FLINK-7490 and FLINK-9220 seem related but too old (legacy planner).

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            nkruber Nico Kruber
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment