[BEAM-702] Simple pattern for per-bundle and per-DoFn Closeable resources

• Type: Improvement
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-ideas
• Assignee: Unassigned
• Reporter: Eugene Kirpichov
• Created: 03/Oct/16 18:45
• Updated: 03/Jun/22 16:16

Description

Dealing with Closeable resources inside a processElement call is easy: simply use try-with-resources.

However, bundle- or DoFn-scoped resources, such as long-lived database connections, are less convenient to deal with: you have to open them in startBundle and conditionally close in finishBundle (likewise setup/teardown), taking special care if there's multiple resources to close all of them.

Perhaps we should provide something like Guava's Closer to DoFn's https://github.com/google/guava/wiki/ClosingResourcesExplained. Ideally, the user would need to only write a startBundle() or setup() method, but not write finishBundle() or teardown() - resources would be closed automatically.