[FLINK-18589] java.io.EOFException when reading integer

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Component/s: Runtime / State Backends

Environment

Starting YARN TaskExecutor runner (Version: 1.10.0, Rev:<unknown>, Date:<unknown>)
OS current user: yarn
Current Hadoop/Kerberos user: hadoop
JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.141-b15
Maximum heap size: 28960 MiBytes
JAVA_HOME: /usr/java/jdk1.8.0_141/jre
Hadoop version: 2.8.5-amzn-6
JVM Options:
 -Xmx30360049728
 -Xms30360049728
 -XX:MaxDirectMemorySize=4429185024
 -XX:MaxMetaspaceSize=1073741824
 -XX:+UseG1GC
 -Dlog.file=/var/log/hadoop-yarn/containers/application_1594539266405_0024/container_1594539266405_0024_01_000050/taskmanager.log
 -Dlog4j.configuration=file:./log4j.properties
Program Arguments:
 -D taskmanager.memory.framework.off-heap.size=134217728b
 -D taskmanager.memory.network.max=1073741824b
 -D taskmanager.memory.network.min=1073741824b
 -D taskmanager.memory.framework.heap.size=134217728b
 -D taskmanager.memory.managed.size=23192823744b
 -D taskmanager.cpu.cores=7.0
 -D taskmanager.memory.task.heap.size=30225832000b
 -D taskmanager.memory.task.off-heap.size=3221225472b
 --configDir .
 -Djobmanager.rpc.address=ip-10-180-28-59.us-west-2.compute.internal
 -Dweb.port=0
 -Dweb.tmpdir=/tmp/flink-web-90514561-0a82-48c4-a8b1-c23bf7d814c0
 -Djobmanager.rpc.port=38887
 -Drest.address=ip-10-180-28-59.us-west-2.compute.internal

• Assignee: Unassigned
• Reporter: Ori Popowski
• Created: 13/Jul/20 16:55
• Updated: 22/Jul/20 08:38
• Resolved: 22/Jul/20 08:36

Description

Getting java.io.EOFException when calling value() on ValueState[java.lang.Integer]

 

Stacktrace:

 

2020-07-13 19:25:11
org.apache.flink.util.FlinkRuntimeException: Error while retrieving data from RocksDB.
	at org.apache.flink.contrib.streaming.state.RocksDBValueState.value(RocksDBValueState.java:92)
	at org.apache.flink.runtime.state.ttl.AbstractTtlDecorator.getWrappedWithTtlCheckAndUpdate(AbstractTtlDecorator.java:92)
	at org.apache.flink.runtime.state.ttl.AbstractTtlDecorator.getWithTtlCheckAndUpdate(AbstractTtlDecorator.java:84)
	at org.apache.flink.runtime.state.ttl.AbstractTtlState.getWithTtlCheckAndUpdate(AbstractTtlState.java:56)
	at org.apache.flink.runtime.state.ttl.TtlValueState.value(TtlValueState.java:44)
	at walkme.flink.TruncateLargeSessions.processElement(TruncateLargeSessions.scala:28)
	at walkme.flink.TruncateLargeSessions.processElement(TruncateLargeSessions.scala:13)
	at org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:85)
	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:173)
	at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.processElement(StreamTaskNetworkInput.java:151)
	at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:128)
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:69)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:311)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:187)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:487)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:470)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException
	at org.apache.flink.core.memory.DataInputDeserializer.readLong(DataInputDeserializer.java:231)
	at org.apache.flink.api.common.typeutils.base.LongSerializer.deserialize(LongSerializer.java:74)
	at org.apache.flink.api.common.typeutils.base.LongSerializer.deserialize(LongSerializer.java:32)
	at org.apache.flink.api.common.typeutils.CompositeSerializer.deserialize(CompositeSerializer.java:151)
	at org.apache.flink.contrib.streaming.state.RocksDBValueState.value(RocksDBValueState.java:90)
	... 18 more

Issue Links

• is related to: FLINK-18637 Key group is not in KeyGroupRange
• relates to: FLINK-18587 java.lang.IllegalArgumentException: Position out of bounds when reading Long state