Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-12319

StackOverFlowError in cep.nfa.sharedbuffer.SharedBuffer

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.6.4, 1.7.2, 1.8.0
    • 1.7.3, 1.8.2, 1.9.0
    • Library / CEP
    • Ubuntu 18.04

      openjdk version "1.8.0_191"
      OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.18.04.1-b12)
      OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

    Description

       

      I wrote a simple SourceFunction that creats Events in a loop.

      The CEP pattern is very simple:

            final Pattern<Event, ?> failurePattern =
                    Pattern.<Event>begin("5 or more failures", AfterMatchSkipStrategy.skipPastLastEvent())
                            .subtype(LoginEvent.class)
                            .where(
                                    new IterativeCondition<LoginEvent>() {
                                        @Override
                                        public boolean filter(LoginEvent value, Context<LoginEvent> ctx) throws Exception {
                                            return value.get("type").equals("failed");
                                        }
                                    })
                            .times(5)
                            .next("1 or more successes")
                            .subtype(LoginEvent.class)
                            .where(
                                    new IterativeCondition<LoginEvent>() {
                                        @Override
                                        public boolean filter(LoginEvent value, Context<LoginEvent> ctx) throws Exception {
                                            return value.get("type").equals("success");
                                        }
                                    })
                            .times(1)
                            .within(Time.milliseconds(20));
      

       

      After about 100k Events, Flink aborts with this stacktrace:

       

      Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
          at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
          at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:630)
          at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
          at org.classdump.alerts.FlinkCep.brute_force_login(FlinkCep.java:263)
          at org.classdump.alerts.FlinkCep.main(FlinkCep.java:41)
      Caused by: java.lang.StackOverflowError
          at org.apache.flink.runtime.state.heap.HeapMapState.get(HeapMapState.java:85)
          at org.apache.flink.runtime.state.UserFacingMapState.get(UserFacingMapState.java:47)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:339)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.releaseNode(SharedBuffer.java:342)
          at org.apache.flink.cep.nfa.sharedbuffer.SharedBuffer.removeNode(SharedBuffer.java:355)
      [...]

       

      This happens with version 1.8.0, 1.7.2, 1.6.4

      Version 1.5.6 does not have this issue.

      Seems to be related to FLINK-9418

       

       

      Attachments

        Issue Links

          Activity

            People

              fan_li_ya Liya Fan
              mpf Marco Pfatschbacher
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m