[BEAM-6857] Support dynamic timers

• Type: New Feature
• Status: Triage Needed
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.20.0
• Component/s: sdk-java-core
• Labels: Done
• Assignee: Rehman Murad Ali
• Reporter: Reuven Lax
• Created: 19/Mar/19 00:34
• Updated: 13/Apr/23 10:59
• Resolved: 12/Feb/20 09:35

Description

The Beam timers API currently requires each timer to be statically specified in the DoFn. The user must provide a separate callback method per timer. For example:

 

DoFn<String, String>()
{   
  @TimerId("timer1") 
  private final TimerSpec timer1 = TimerSpecs.timer(...);   
  @TimerId("timer2") 
  private final TimerSpec timer2 = TimerSpecs.timer(...);                 
  ...... set timers in processElement    
  @OnTimer("timer1") 
  public void onTimer1() { .....}
  @OnTimer("timer2") 
  public void onTimer2() {....}
}

 

However there are many cases where the user does not know the set of timers statically when writing their code. This happens when the timer tag should be based on the data. It also happens when writing a DSL on top of Beam, where the DSL author has to create DoFns but does not know statically which timers their users will want to set (e.g. Scio).

 

The goal is to support dynamic timers. Something as follows;

 

DoFn<String, String>() 
{
  @TimerId("timer") 
  private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
  @ProcessElement process(@TimerId("timer") DynamicTimer timer)
  {
       timer.set("tag1'", ts);       
       timer.set("tag2", ts);     
  }
  @OnTimer("timer") 
  public void onTimer1(@TimerTag String tag) { .....}
}

 

Issue Links

• is related to: BEAM-9603 Support Dynamic Timer in Java SDK over FnApi
• is related to: BEAM-9602 Support Dynamic Timer in Python SDK over FnApi
• relates to: BEAM-9912 Support Dynamic Timers in the Spark Runner
• relates to: BEAM-10120 Support Dynamic Timers in the Flink Portable Runner
• links to: GitHub Pull Request #10315
• links to: GitHub Pull Request #10316
• links to: GitHub Pull Request #10506
• links to: GitHub Pull Request #10747
• links to: GitHub Pull Request #10871