Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-4651

Re-register processing time timers at the WindowOperator upon recovery.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0, 1.1.5
    • Component/s: DataStream API
    • Labels:

      Description

      Currently the WindowOperator checkpoints the processing time timers, but upon recovery it does not re-registers them with the TimeServiceProvider. To actually reprocess them it relies on another element that will come and register a new timer for a future point in time. Although this is a realistic assumption in long running jobs, we can remove this assumption by re-registering the restored timers with the TimeServiceProvider in the open() method of the WindowOperator.

        Activity

        Hide
        StephanEwen Stephan Ewen added a comment -

        Fixed in 1.2 as part of 47e4977442f7045baf29ce7dc772b7f7aff65343
        1.1.3 does not have a fix for that.

        Show
        StephanEwen Stephan Ewen added a comment - Fixed in 1.2 as part of 47e4977442f7045baf29ce7dc772b7f7aff65343 1.1.3 does not have a fix for that.
        Hide
        StephanEwen Stephan Ewen added a comment -

        How will this be resolved?

        Show
        StephanEwen Stephan Ewen added a comment - How will this be resolved?
        Hide
        aljoscha Aljoscha Krettek added a comment -

        We can either back port the "only one physical processing-time timer" change or only make sure that we register timers in open().

        Show
        aljoscha Aljoscha Krettek added a comment - We can either back port the "only one physical processing-time timer" change or only make sure that we register timers in open() .
        Hide
        StephanEwen Stephan Ewen added a comment -

        The big changes in the window operator (move timers, key groups) came after that patch, I think. So we can probably backport it without too much effort.

        Show
        StephanEwen Stephan Ewen added a comment - The big changes in the window operator (move timers, key groups) came after that patch, I think. So we can probably backport it without too much effort.
        Hide
        uce Ufuk Celebi added a comment -

        Kostas Kloudas What's the state of this? Can we get it in for 1.1.4?

        Show
        uce Ufuk Celebi added a comment - Kostas Kloudas What's the state of this? Can we get it in for 1.1.4?
        Hide
        uce Ufuk Celebi added a comment -

        Talked to Aljoscha Krettek about back porting this, I think he didn't have time to finish it yet. Won't block RC2 on this.

        Show
        uce Ufuk Celebi added a comment - Talked to Aljoscha Krettek about back porting this, I think he didn't have time to finish it yet. Won't block RC2 on this.
        Hide
        skidder Scott Kidder added a comment -

        I can confirm that this issue appears to be fixed in the release-1.2.0-rc0 branch (commit f3c59cedae7a508825d8032a8aa9f5af6d177555).

        Show
        skidder Scott Kidder added a comment - I can confirm that this issue appears to be fixed in the release-1.2.0-rc0 branch (commit f3c59cedae7a508825d8032a8aa9f5af6d177555).

          People

          • Assignee:
            kkl0u Kostas Kloudas
            Reporter:
            kkl0u Kostas Kloudas
          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development