[FLINK-6935] Integration of SQL and CEP

• Type: New Feature
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Library / CEP, Table SQL / API
• Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
• Assignee: Unassigned
• Reporter: Jark Wu
• Created: 16/Jun/17 08:03
• Updated: 25/Aug/23 10:35

Description

Flink's CEP library is a great library for complex event processing, more and more customers are expressing their interests in it. But it also has some limitations that users usually have to write a lot of code even for a very simple pattern match use case as it currently only supports the Java API.

CEP DSLs and SQLs strongly resemble each other. CEP's additional features compared to SQL boil down to pattern detection. So It will be awesome to consolidate CEP and SQL. It makes SQL more powerful to support more usage scenario. And it gives users the ability to easily and quickly to build CEP applications.

The FLIP can be found here:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-20%3A+Integration+of+SQL+and+CEP

This is an umbrella issue for the FLIP. We should wait for Calcite 1.13 to start this work.

Issue Links

• is blocked by: FLINK-6429 Bump up Calcite version to 1.13
• requires: FLINK-10590 Optional quantifier is discarded from first element of group pattern
• mentioned in: Page Loading...

Sub-Tasks

• Support partition by and order by of MATCH_RECOGNIZE: Closed
• Fix EMPTY MATCH bug in CEP.: Closed
• Support pattern group in CEP: Resolved
• Support greedy quantifier in CEP: Resolved
• Refactor NFACompiler to reduce code duplication: Resolved
• Not store IterativeCondition with NFA state: Resolved
• Support for quantifier range to CEP's pattern API: Resolved
• Support custom order by in PatternStream: Closed
• Support AFTER MATCH SKIP function in CEP library API: Resolved
• Add method to check if pattern can produce empty matches: Resolved
• Support the basic functionality of MATCH_RECOGNIZE: Resolved
• IterativeCondition should support RichFunction interface: Closed
• Pattern(Flat)SelectFunctions should support RichFunction interface: Closed
• Enable UDFs support in MATCH_RECOGNIZE: Closed
• Add access to timerService in Pattern(Flat)SelectFunction: Closed
• Add functions to return TimeIndicators from MATCH_RECOGNIZE: Closed
• Support aggregation functions in the define and measures clause of MatchRecognize: Closed
• Add support for ALL ROWS PER MATCH: Open
• Add support for CLASSIFIER() function in MatchRecognize: Open
• Improve support for greedy/reluctant quantifiers: Open
• Group patterns should support greedy quantifiers: Open
• Support time constraint in MatchRecognize: Closed
• Support SUBSETS: In Progress
• Support alterations in patterns: In Progress
• Support exclusions in patterns: Open
• Support reusing same variable in Pattern: Open
• Support physical offset in MATCH_RECOGNIZE (PREV): Open
• Extend streaming SQL end-to-end test to test MATCH_RECOGNIZE: Resolved
• Add MATCH_RECOGNIZE documentation: Closed
• Document MATCH_RECOGNIZE: Closed
• Support patterns that can produce empty matches: Open
• Support DISTINCT aggregates in MATCH_RECOGNIZE: Open
• Fix the bug that fix time quantifier can not be the last element of a pattern: Closed
• Support MATCH_RECOGNIZE in Batch mode: Closed