[BEAM-12220] ZipFiles.zipDirectory leaks native JVM memory

• Type: Bug
• Status: Triage Needed
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.22.0
• Fix Version/s: 2.30.0
• Component/s: runner-core, runner-dataflow, sdk-java-core
• Assignee: Brian Hulette
• Reporter: Brian Hulette
• Created: 23/Apr/21 22:09
• Updated: 13/Apr/23 11:05
• Resolved: 27/Apr/21 09:52

Description

Noticed this in a long-running JVM process starting multiple Dataflow pipelines. Heap usage slowly rose as observed by jcmd <pid> VM.native_memory summary.diff, but a heap dump did not show a consistent amount of usage.

Followed this guide for identify native memory leaks, which pointed me to java.util.ZipFile.

We use it in runners.core-construction.PipelineResources via sdk.util.ZipFiles

Closing the ZipOutputStream here seems to alleviate the issue.

Issue Links

• links to: GitHub Pull Request #14631
• links to: GitHub Pull Request #14654