Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6208

Implement skip till next match strategy

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: CEP
    • Labels:
      None

      Description

      Right now, we support two strategies (except for looping states):

      • skip till any match -> followedBy
      • strict contiguity -> next

      We should also support a strategy that will match only the first occurence of a valid pattern.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user dawidwys opened a pull request:

          https://github.com/apache/flink/pull/3698

          FLINK-6208 Implement skip till next match strategy

          Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
          If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
          In addition to going through the list, please provide a meaningful description of your changes.

          • [ ] General
          • The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
          • The pull request addresses only one issue
          • Each commit in the PR has a meaningful commit message (including the JIRA id)
          • [ ] Documentation
          • Documentation has been added for new functionality
          • Old documentation affected by the pull request has been updated
          • JavaDoc for public methods has been added
          • [ ] Tests & Build
          • Functionality added by the pull request is covered by tests
          • `mvn clean verify` has been executed successfully locally or a Travis build has passed

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/dawidwys/flink cep-skip-till-next

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3698.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3698


          commit 362a3889d36ebf96469653d4c94bf2494bc49847
          Author: Dawid Wysakowicz <dawid@getindata.com>
          Date: 2017-04-07T12:45:40Z

          FLINK-6208 Implement skip till next match strategy


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user dawidwys opened a pull request: https://github.com/apache/flink/pull/3698 FLINK-6208 Implement skip till next match strategy Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration. If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide] ( http://flink.apache.org/how-to-contribute.html ). In addition to going through the list, please provide a meaningful description of your changes. [ ] General The pull request references the related JIRA issue (" [FLINK-XXX] Jira title text") The pull request addresses only one issue Each commit in the PR has a meaningful commit message (including the JIRA id) [ ] Documentation Documentation has been added for new functionality Old documentation affected by the pull request has been updated JavaDoc for public methods has been added [ ] Tests & Build Functionality added by the pull request is covered by tests `mvn clean verify` has been executed successfully locally or a Travis build has passed You can merge this pull request into a Git repository by running: $ git pull https://github.com/dawidwys/flink cep-skip-till-next Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3698.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3698 commit 362a3889d36ebf96469653d4c94bf2494bc49847 Author: Dawid Wysakowicz <dawid@getindata.com> Date: 2017-04-07T12:45:40Z FLINK-6208 Implement skip till next match strategy
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user kl0u commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3698#discussion_r113663695

          — Diff: flink-libraries/flink-cep/src/test/java/org/apache/flink/cep/nfa/NFAITCase.java —
          @@ -1609,7 +1549,7 @@ public boolean filter(Event value) throws Exception {

          compareMaps(resultingPatterns, Lists.<List<Event>>newArrayList(
          Lists.newArrayList(ConsecutiveData.startEvent, ConsecutiveData.middleEvent1, ConsecutiveData.middleEvent2, ConsecutiveData.end),

          • Lists.newArrayList(ConsecutiveData.startEvent, ConsecutiveData.middleEvent2, ConsecutiveData.middleEvent3, ConsecutiveData.end)
            + Lists.newArrayList(ConsecutiveData.startEvent, ConsecutiveData.middleEvent1, ConsecutiveData.middleEvent3, ConsecutiveData.end)
              • End diff –

          Here the expected result should only be the first pattern, because we are in `consecutive` mode.
          The problem is the

          ```
          if (quantifier.hasProperty(Quantifier.QuantifierProperty.CONSECUTIVE) &&
          !quantifier.hasProperty(Quantifier.QuantifierProperty.EAGER)) {
          //inner branching and no invalid elements in between
          return (IterativeCondition<T>) pattern.getCondition();
          ```
          in the `getInnerIgnoreCondition()` of the `NFACompiler`.

          Show
          githubbot ASF GitHub Bot added a comment - Github user kl0u commented on a diff in the pull request: https://github.com/apache/flink/pull/3698#discussion_r113663695 — Diff: flink-libraries/flink-cep/src/test/java/org/apache/flink/cep/nfa/NFAITCase.java — @@ -1609,7 +1549,7 @@ public boolean filter(Event value) throws Exception { compareMaps(resultingPatterns, Lists.<List<Event>>newArrayList( Lists.newArrayList(ConsecutiveData.startEvent, ConsecutiveData.middleEvent1, ConsecutiveData.middleEvent2, ConsecutiveData.end), Lists.newArrayList(ConsecutiveData.startEvent, ConsecutiveData.middleEvent2, ConsecutiveData.middleEvent3, ConsecutiveData.end) + Lists.newArrayList(ConsecutiveData.startEvent, ConsecutiveData.middleEvent1, ConsecutiveData.middleEvent3, ConsecutiveData.end) End diff – Here the expected result should only be the first pattern, because we are in `consecutive` mode. The problem is the ``` if (quantifier.hasProperty(Quantifier.QuantifierProperty.CONSECUTIVE) && !quantifier.hasProperty(Quantifier.QuantifierProperty.EAGER)) { //inner branching and no invalid elements in between return (IterativeCondition<T>) pattern.getCondition(); ``` in the `getInnerIgnoreCondition()` of the `NFACompiler`.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user kl0u commented on the issue:

          https://github.com/apache/flink/pull/3698

          Thanks a lot for the work @dawidwys ! I really like the changes. The `NFACompiler` seems cleaner and the removal of the `FollowedByPattern` class makes the code nicer. I will wait for travis and then merge.

          Show
          githubbot ASF GitHub Bot added a comment - Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3698 Thanks a lot for the work @dawidwys ! I really like the changes. The `NFACompiler` seems cleaner and the removal of the `FollowedByPattern` class makes the code nicer. I will wait for travis and then merge.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user kl0u commented on the issue:

          https://github.com/apache/flink/pull/3698

          Hi @dawidwys. Thanks for the work.
          I merged this.
          Could you close the PR and the related JIRA?

          Show
          githubbot ASF GitHub Bot added a comment - Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3698 Hi @dawidwys. Thanks for the work. I merged this. Could you close the PR and the related JIRA?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user dawidwys closed the pull request at:

          https://github.com/apache/flink/pull/3698

          Show
          githubbot ASF GitHub Bot added a comment - Github user dawidwys closed the pull request at: https://github.com/apache/flink/pull/3698

            People

            • Assignee:
              dawidwys Dawid Wysakowicz
              Reporter:
              dawidwys Dawid Wysakowicz
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development