Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6614

Applying function on window auxiliary function fails

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 1.4.0
    • Fix Version/s: 1.3.0, 1.4.0
    • Component/s: Table API & SQL
    • Labels:
      None

      Description

      SQL queries that apply a function or expression on a window auxiliary function (TUMBLE_START, TUMBLE_END, HOP_START, etc). cannot be translated and fail with a CodeGenException:

      Exception in thread "main" org.apache.flink.table.codegen.CodeGenException: Unsupported call: TUMBLE_END
      

      Example query:

      SELECT 
        a, 
        toLong(TUMBLE_END(rowtime, INTERVAL '10' MINUTE)) AS t, 
        COUNT(b) AS cntB
      FROM myTable
      GROUP BY a, TUMBLE(rowtime, INTERVAL '10' MINUTE)
      

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user fhueske opened a pull request:

          https://github.com/apache/flink/pull/3930

          FLINK-6614 [table] Fix translation of group auxiliary functions (e.g., TUMBLE_END).

          When translating group auxiliary functions, we only check the root expressions of a projection but do not dig into `RexCall`s and check if these functions are used in their operands.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/fhueske/flink tableAuxExpr

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3930.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3930



          Show
          githubbot ASF GitHub Bot added a comment - GitHub user fhueske opened a pull request: https://github.com/apache/flink/pull/3930 FLINK-6614 [table] Fix translation of group auxiliary functions (e.g., TUMBLE_END). When translating group auxiliary functions, we only check the root expressions of a projection but do not dig into `RexCall`s and check if these functions are used in their operands. You can merge this pull request into a Git repository by running: $ git pull https://github.com/fhueske/flink tableAuxExpr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3930.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3930
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user sunjincheng121 commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3930#discussion_r117205976

          — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala —
          @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase

          { streamUtil.verifySql(sql, expected) }

          + @Test
          + def testExpressionOnWindowAuxFunction() = {
          + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge)
          — End diff –

          We can remove this line or using the `weightedAvg` in sql.

          Show
          githubbot ASF GitHub Bot added a comment - Github user sunjincheng121 commented on a diff in the pull request: https://github.com/apache/flink/pull/3930#discussion_r117205976 — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala — @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase { streamUtil.verifySql(sql, expected) } + @Test + def testExpressionOnWindowAuxFunction() = { + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge) — End diff – We can remove this line or using the `weightedAvg` in sql.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3930#discussion_r117220211

          — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala —
          @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase

          { streamUtil.verifySql(sql, expected) }

          + @Test
          + def testExpressionOnWindowAuxFunction() = {
          + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge)
          — End diff –

          good point, thanks!
          I'll remove it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on a diff in the pull request: https://github.com/apache/flink/pull/3930#discussion_r117220211 — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala — @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase { streamUtil.verifySql(sql, expected) } + @Test + def testExpressionOnWindowAuxFunction() = { + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge) — End diff – good point, thanks! I'll remove it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on the issue:

          https://github.com/apache/flink/pull/3930

          Merging

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/3930 Merging
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3930

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3930
          Hide
          fhueske Fabian Hueske added a comment -

          Fixed for 1.3.0 with 51fb7ed791bc8c5c1c35dffcd9855a2e5a8f3087
          Fixed for 1.4.0 with 9fc42df68d746c633b0d3c8995e0031064bfd362

          Show
          fhueske Fabian Hueske added a comment - Fixed for 1.3.0 with 51fb7ed791bc8c5c1c35dffcd9855a2e5a8f3087 Fixed for 1.4.0 with 9fc42df68d746c633b0d3c8995e0031064bfd362

            People

            • Assignee:
              fhueske Fabian Hueske
              Reporter:
              fhueske Fabian Hueske
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development