Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6614

Applying function on window auxiliary function fails

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 1.3.0, 1.4.0
        • Fix Version/s: 1.3.0, 1.4.0
        • Component/s: Table API & SQL
        • Labels:
          None

          Description

          SQL queries that apply a function or expression on a window auxiliary function (TUMBLE_START, TUMBLE_END, HOP_START, etc). cannot be translated and fail with a CodeGenException:

          Exception in thread "main" org.apache.flink.table.codegen.CodeGenException: Unsupported call: TUMBLE_END

          Example query:

          SELECT 
  a, 
  toLong(TUMBLE_END(rowtime, INTERVAL '10' MINUTE)) AS t, 
  COUNT(b) AS cntB
FROM myTable
GROUP BY a, TUMBLE(rowtime, INTERVAL '10' MINUTE)

            Issue Links

            links to

            Web Link GitHub Pull Request #3930

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              GitHub user fhueske opened a pull request:

              https://github.com/apache/flink/pull/3930

              FLINK-6614 [table] Fix translation of group auxiliary functions (e.g., TUMBLE_END).

              When translating group auxiliary functions, we only check the root expressions of a projection but do not dig into `RexCall`s and check if these functions are used in their operands.

              You can merge this pull request into a Git repository by running:

              $ git pull https://github.com/fhueske/flink tableAuxExpr

              Alternatively you can review and apply these changes as the patch at:

              https://github.com/apache/flink/pull/3930.patch

              To close this pull request, make a commit to your master/trunk branch
              with (at least) the following in the commit message:

              This closes #3930

              Show
              githubbot ASF GitHub Bot added a comment - GitHub user fhueske opened a pull request: https://github.com/apache/flink/pull/3930 FLINK-6614 [table] Fix translation of group auxiliary functions (e.g., TUMBLE_END). When translating group auxiliary functions, we only check the root expressions of a projection but do not dig into `RexCall`s and check if these functions are used in their operands. You can merge this pull request into a Git repository by running: $ git pull https://github.com/fhueske/flink tableAuxExpr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3930.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3930
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              Github user sunjincheng121 commented on a diff in the pull request:

              https://github.com/apache/flink/pull/3930#discussion_r117205976

              — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala —
              @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase

              { streamUtil.verifySql(sql, expected) }

              + @Test
              + def testExpressionOnWindowAuxFunction() = {
              + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge)
              — End diff –

              We can remove this line or using the `weightedAvg` in sql.

              Show
              githubbot ASF GitHub Bot added a comment - Github user sunjincheng121 commented on a diff in the pull request: https://github.com/apache/flink/pull/3930#discussion_r117205976 — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala — @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase { streamUtil.verifySql(sql, expected) } + @Test + def testExpressionOnWindowAuxFunction() = { + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge) — End diff – We can remove this line or using the `weightedAvg` in sql.
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              Github user fhueske commented on a diff in the pull request:

              https://github.com/apache/flink/pull/3930#discussion_r117220211

              — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala —
              @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase

              { streamUtil.verifySql(sql, expected) }

              + @Test
              + def testExpressionOnWindowAuxFunction() = {
              + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge)
              — End diff –

              good point, thanks!
              I'll remove it.

              Show
              githubbot ASF GitHub Bot added a comment - Github user fhueske commented on a diff in the pull request: https://github.com/apache/flink/pull/3930#discussion_r117220211 — Diff: flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/scala/stream/sql/WindowAggregateTest.scala — @@ -150,6 +150,35 @@ class WindowAggregateTest extends TableTestBase { streamUtil.verifySql(sql, expected) } + @Test + def testExpressionOnWindowAuxFunction() = { + streamUtil.tEnv.registerFunction("weightedAvg", new WeightedAvgWithMerge) — End diff – good point, thanks! I'll remove it.
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              Github user fhueske commented on the issue:

              https://github.com/apache/flink/pull/3930

              Merging

              Show
              githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/3930 Merging
              Hide
              Permalink
              githubbot ASF GitHub Bot added a comment -

              Github user asfgit closed the pull request at:

              https://github.com/apache/flink/pull/3930

              Show
              githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3930
              Hide
              Permalink
              fhueske Fabian Hueske added a comment -

              Fixed for 1.3.0 with 51fb7ed791bc8c5c1c35dffcd9855a2e5a8f3087
              Fixed for 1.4.0 with 9fc42df68d746c633b0d3c8995e0031064bfd362

              Show
              fhueske Fabian Hueske added a comment - Fixed for 1.3.0 with 51fb7ed791bc8c5c1c35dffcd9855a2e5a8f3087 Fixed for 1.4.0 with 9fc42df68d746c633b0d3c8995e0031064bfd362

                People

                • Assignee:
                  fhueske Fabian Hueske
                  Reporter:
                  fhueske Fabian Hueske
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development