Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6780

ExternalTableSource should add time attributes in the row type

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.1, 1.4.0
    • Component/s: Table API & SQL
    • Labels:
      None

      Description

      We observed that all streaming queries that refer to external tables fail when the Volcano planner converting LogicalTableScan to FlinkLogicalTableSourceScan:

      Type mismatch:
      rowtype of new rel:
      RecordType(<table schema>, TIMESTAMP(3) NOT NULL proctime) NOT NULL
      rowtype of set:
      RecordType(<table schema>, ...) NOT NULL
      

      Tables that are registered through StreamTableEnvironment#registerTableSource() do not suffer from this problem as StreamTableSourceTable adds the processing time / event time attribute automatically.

        Issue Links

          Activity

          Hide
          sunjincheng121 sunjincheng added a comment -

          HI, Haohui Mai If I understand you correctly, your meant currently the external table source not implement both DefinedRowtimeAttribute and DefinedProctimeAttribute interface. e.g: CsvTableSource, when we using CsvTableSource by register registerExternalCatalog, FlinkLogicalTableSourceScan#deriveRowType will get None XXXtimeAttribute. Please tell me if i understand you incorrectly.

          And it's great if you can add test case for the error above.

          Thanks,
          SunJincheng

          Show
          sunjincheng121 sunjincheng added a comment - HI, Haohui Mai If I understand you correctly, your meant currently the external table source not implement both DefinedRowtimeAttribute and DefinedProctimeAttribute interface. e.g: CsvTableSource , when we using CsvTableSource by register registerExternalCatalog , FlinkLogicalTableSourceScan#deriveRowType will get None XXXtimeAttribute . Please tell me if i understand you incorrectly. And it's great if you can add test case for the error above. Thanks, SunJincheng
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user haohui opened a pull request:

          https://github.com/apache/flink/pull/4023

          FLINK-6780 [table] ExternalTableSource should add time attributes in the row type

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/haohui/flink FLINK-6780

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/4023.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #4023


          commit b06cf4f897dc37f1beb47f4bac38339fb22751ba
          Author: Haohui Mai <wheat9@apache.org>
          Date: 2017-05-31T08:40:53Z

          FLINK-6780 [table] ExternalTableSource should add time attributes in the row type.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user haohui opened a pull request: https://github.com/apache/flink/pull/4023 FLINK-6780 [table] ExternalTableSource should add time attributes in the row type You can merge this pull request into a Git repository by running: $ git pull https://github.com/haohui/flink FLINK-6780 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4023.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4023 commit b06cf4f897dc37f1beb47f4bac38339fb22751ba Author: Haohui Mai <wheat9@apache.org> Date: 2017-05-31T08:40:53Z FLINK-6780 [table] ExternalTableSource should add time attributes in the row type.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user twalthr commented on the issue:

          https://github.com/apache/flink/pull/4023

          Thanks @haohui. The change looks good. I will merge this.

          Show
          githubbot ASF GitHub Bot added a comment - Github user twalthr commented on the issue: https://github.com/apache/flink/pull/4023 Thanks @haohui. The change looks good. I will merge this.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/4023

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4023
          Hide
          twalthr Timo Walther added a comment -

          Fixed in 1.4: 1beab38134a77ccebe35ea2c199bce5c13db8c60
          Fixed in 1.3: 5f53052f676ebf37de2d64cd0809704da16fe246

          Show
          twalthr Timo Walther added a comment - Fixed in 1.4: 1beab38134a77ccebe35ea2c199bce5c13db8c60 Fixed in 1.3: 5f53052f676ebf37de2d64cd0809704da16fe246
          Hide
          rmetzger Robert Metzger added a comment -

          Fix Version/s says 1.3.0, but that's not true, right?

          Show
          rmetzger Robert Metzger added a comment - Fix Version/s says 1.3.0, but that's not true, right?

            People

            • Assignee:
              wheat9 Haohui Mai
              Reporter:
              wheat9 Haohui Mai
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development