[BEAM-2831] Pipeline crashes due to Beam encoder breaking Flink memory management

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.0.0, 2.1.0
• Fix Version/s: 2.5.0
• Component/s: runner-flink
• Environment: Flink 1.2.1 and 1.3.0, Java HotSpot and OpenJDK 8, macOS 10.12.6 and unknown Linux
• External issue URL: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/EOFException-related-to-memory-segments-during-run-of-Beam-pipeline-on-Flink-td15255.html
• Assignee: Dawid Wysakowicz
• Reporter: Reinier Kip
• Created: 31/Aug/17 11:53
• Updated: 16/May/20 13:42
• Resolved: 27/Mar/18 17:57

Description

I’ve been running a Beam pipeline on Flink. Depending on the dataset size and the heap memory configuration of the jobmanager and taskmanager, I may run into an EOFException, which causes the job to fail.

As discussed on Flink's mailinglist (stacktrace enclosed), Flink catches these EOFExceptions and activates disk spillover. Because Beam wraps these exceptions, this mechanism fails, the exception travels up the stack, and the job aborts.

Hopefully this is enough information and this is something that can be adjusted for in Beam. I'd be glad to provide more information where needed.

Issue Links

• links to: GitHub Pull Request #4892