[BEAM-5307] Allow injection of state in DoFn.FinishBundle

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Duplicate
• Fix Version/s: Not applicable
• Component/s: sdk-java-core
• Labels: Done
• Assignee: Kenneth Knowles
• Reporter: Sander Ploegsma
• Created: 05/Sep/18 09:45
• Updated: 06/Oct/20 21:55
• Resolved: 23/Oct/18 20:48

Description

Example use case: a stateful DoFn that requires persisting its state to an external database. Instead of writing to the external database for each element, it is much more efficient to flush the state every once in a while, or when cleaning up.

Currently, this is not possible because the @StateId injection is only available in processing functions and timers. This might be a workaround, but I'm not even sure if it works.

Instead, by allowing the use of @StateId inside a @FinishBundle function, we can make sure the internal state is persisted in all scenarios:

@FinishBundle
public void flush(FinishBundleContext context, @StateId("myState") ValueState<StateObj> state) {
    repository.save(state.read());
}

 

Issue Links

• duplicates: BEAM-1589 Add OnWindowExpiration method to Stateful DoFn