Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10211

Time indicators are not correctly materialized for LogicalJoin

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      Currently org.apache.flink.table.calcite.RelTimeIndicatorConverter#visit(LogicalJoin) correctly handles only windowed joins. Output of non windowed joins shouldn't contain any time indicators.

      A symptom of this is the exception:

      Rowtime attributes must not be in the input rows of a regular join. As a workaround you can cast the time attributes of input tables to TIMESTAMP before.
      

      Or this exception:

      org.apache.flink.table.api.TableException: Found more than one rowtime field: [orderTime, payTime] in the table that should be converted to a DataStream. Please select the rowtime field that should be used as event-time timestamp for the DataStream by casting all other fields to TIMESTAMP. at org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:906)
      
      

      A long-term solution would be:

      The root cause of this issue is the early phase in which RelTimeIndicatorConverter is called. Due to lack of information (since the join condition might not have been pushed into the join node), we can not differentiate between a window and non-window join. Thus, we cannot perform the time indicator materialization more fine grained. A solution would be to perform the materialization later after the logical optimization and before the physical translation, this would also make sense from a semantic perspective because time indicators are more a physical characteristic.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              pnowojski Piotr Nowojski

              Dates

              • Created:
                Updated:

                Issue deployment