Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-6857

Support dynamic timers

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.20.0
    • Component/s: sdk-java-core
    • Labels:

      Description

      The Beam timers API currently requires each timer to be statically specified in the DoFn. The user must provide a separate callback method per timer. For example:

       

      DoFn<String, String>()
      {   
        @TimerId("timer1") 
        private final TimerSpec timer1 = TimerSpecs.timer(...);   
        @TimerId("timer2") 
        private final TimerSpec timer2 = TimerSpecs.timer(...);                 
        ...... set timers in processElement    
        @OnTimer("timer1") 
        public void onTimer1() { .....}
        @OnTimer("timer2") 
        public void onTimer2() {....}
      }
      

       

      However there are many cases where the user does not know the set of timers statically when writing their code. This happens when the timer tag should be based on the data. It also happens when writing a DSL on top of Beam, where the DSL author has to create DoFns but does not know statically which timers their users will want to set (e.g. Scio).

       

      The goal is to support dynamic timers. Something as follows;

       

      DoFn<String, String>() 
      {
        @TimerId("timer") 
        private final TimerSpec timer1 = TimerSpecs.dynamicTimer(...);
        @ProcessElement process(@TimerId("timer") DynamicTimer timer)
        {
             timer.set("tag1'", ts);       
             timer.set("tag2", ts);     
        }
        @OnTimer("timer") 
        public void onTimer1(@TimerTag String tag) { .....}
      }
      

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rehmanmuradali Rehman Murad Ali
                Reporter:
                reuvenlax Reuven Lax
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 31h 10m
                  31h 10m