Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-22962

Key group is not in KeyGroupRange error while checkpointing

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.12.1
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      Linux

      Description

      Hi,

       

      We are getting the below exception while using rocksdb as state backend at the time of checkpointing:

      2021-06-10 12:05:13,933 INFO org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable [] - Aggregator (3/4)#0 - asynchronous part of checkpoint 2 could not be completed.
      java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Key group 0 is not in KeyGroupRange{startKeyGroup=5, endKeyGroup=7}.
      at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_261]
      at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_261]
      at org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:621) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.<init>(OperatorSnapshotFinalizer.java:54) ~[flink-streaming-java_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:122) [flink-streaming-java_2.11-1.12.1.jar:1.12.1]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
      at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
      Caused by: java.lang.IllegalArgumentException: Key group 0 is not in KeyGroupRange{startKeyGroup=5, endKeyGroup=7}.
      at org.apache.flink.runtime.state.KeyGroupRangeOffsets.computeKeyGroupIndex(KeyGroupRangeOffsets.java:144) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.runtime.state.KeyGroupRangeOffsets.setKeyGroupOffset(KeyGroupRangeOffsets.java:106) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.writeKVStateData(RocksFullSnapshotStrategy.java:333) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.writeSnapshotToOutputStream(RocksFullSnapshotStrategy.java:264) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.callInternal(RocksFullSnapshotStrategy.java:227) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.contrib.streaming.state.snapshot.RocksFullSnapshotStrategy$SnapshotAsynchronousPartCallable.callInternal(RocksFullSnapshotStrategy.java:180) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at org.apache.flink.runtime.state.AsyncSnapshotCallable.call(AsyncSnapshotCallable.java:78) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_261]
      at org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:618) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
      ... 5 more

       

      When we change the state backend to file or heap we do not get this error.

       

       

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              prateekkohli2112 Prateek Kohli
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: