Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5450

WindowOperator logs about "re-registering state from an older Flink version" even though its not a restored window

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.0, 1.3.0
    • Component/s: DataStream API
    • Labels:
      None

      Description

      While testing the RC0 of Flink 1.2, I stumbled across this log message

      15:42:02,855 INFO  org.apache.flink.streaming.api.operators.AbstractStreamOperator  - WindowOperator (taskIdx=WindowOperator) re-registering state from an older Flink version.
      

      My WindowOperator is not restored, so I find this log message a bit misleading.

        Issue Links

          Activity

          Hide
          Zentol Chesnay Schepler added a comment -

          It is also malformatted; the log message has 2 wildcards but 3 arguments.

          Show
          Zentol Chesnay Schepler added a comment - It is also malformatted; the log message has 2 wildcards but 3 arguments.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user kl0u opened a pull request:

          https://github.com/apache/flink/pull/3134

          FLINK-5450 Fix restore from legacy log message

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/kl0u/flink window-op-logging

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3134.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3134


          commit 664a3b3270bd427b8d449b8abf551d1da5e151fd
          Author: kl0u <kkloudas@gmail.com>
          Date: 2017-01-16T14:27:38Z

          FLINK-5450 Fix restore from legacy log message


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user kl0u opened a pull request: https://github.com/apache/flink/pull/3134 FLINK-5450 Fix restore from legacy log message You can merge this pull request into a Git repository by running: $ git pull https://github.com/kl0u/flink window-op-logging Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3134.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3134 commit 664a3b3270bd427b8d449b8abf551d1da5e151fd Author: kl0u <kkloudas@gmail.com> Date: 2017-01-16T14:27:38Z FLINK-5450 Fix restore from legacy log message
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user StephanEwen commented on the issue:

          https://github.com/apache/flink/pull/3134

          To understand the implications better: Is this only logged on the initial legacy savepoint restore? Or also on subsequent checkpoint restore operations?

          Show
          githubbot ASF GitHub Bot added a comment - Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3134 To understand the implications better: Is this only logged on the initial legacy savepoint restore? Or also on subsequent checkpoint restore operations?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user kl0u commented on the issue:

          https://github.com/apache/flink/pull/3134

          From what I understood from the JIRA, it is only when restoring from legacy. If not, then the problem is deeper. @rmetzger ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3134 From what I understood from the JIRA, it is only when restoring from legacy. If not, then the problem is deeper. @rmetzger ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on the issue:

          https://github.com/apache/flink/pull/3134

          Quick side note: There was a discussion on the mailing list about not using the `R: @somebody` notes anymore. Would be cool if you chime in there if you disagree with that and want to continue using it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on the issue: https://github.com/apache/flink/pull/3134 Quick side note: There was a discussion on the mailing list about not using the `R: @somebody` notes anymore. Would be cool if you chime in there if you disagree with that and want to continue using it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3134

          (To quickly respond to your question @kl0u: The original problem was that the log message has been logged every time when starting a window operator. Even when there was no restore or anything happening. I think Stephan's question was about the implications of your change)

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3134 (To quickly respond to your question @kl0u: The original problem was that the log message has been logged every time when starting a window operator. Even when there was no restore or anything happening. I think Stephan's question was about the implications of your change)
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user kl0u commented on the issue:

          https://github.com/apache/flink/pull/3134

          @rmetzger I see. Then my PR fixes it, as the problem was that when retrieving timers, we were first putting them in the temporary list in the `initializeState` and the re-register them with the `TimerService` in the `open()`. When re-registering we were not checking if there were any timers to re-register before printing the message. That is why the message was logged all the time.

          Show
          githubbot ASF GitHub Bot added a comment - Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3134 @rmetzger I see. Then my PR fixes it, as the problem was that when retrieving timers, we were first putting them in the temporary list in the `initializeState` and the re-register them with the `TimerService` in the `open()`. When re-registering we were not checking if there were any timers to re-register before printing the message. That is why the message was logged all the time.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aljoscha commented on the issue:

          https://github.com/apache/flink/pull/3134

          Just to also give my 2 cents: the changes look good. 👍

          Show
          githubbot ASF GitHub Bot added a comment - Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/3134 Just to also give my 2 cents: the changes look good. 👍
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3134

          Thank you. I'll merge it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3134 Thank you. I'll merge it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user kl0u commented on the issue:

          https://github.com/apache/flink/pull/3134

          Thanks a lot!

          Show
          githubbot ASF GitHub Bot added a comment - Github user kl0u commented on the issue: https://github.com/apache/flink/pull/3134 Thanks a lot!
          Show
          rmetzger Robert Metzger added a comment - Resolved in http://git-wip-us.apache.org/repos/asf/flink/commit/f4cd7d0e for 1.2 http://git-wip-us.apache.org/repos/asf/flink/commit/f3419af3 for 1.3
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3134

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3134
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user StephanEwen commented on the issue:

          https://github.com/apache/flink/pull/3134

          I think this should go to master and Flink-1.2

          Show
          githubbot ASF GitHub Bot added a comment - Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3134 I think this should go to master and Flink-1.2
          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3134 I've merged it to 1.2 and 1.3: https://issues.apache.org/jira/browse/FLINK-5450?focusedCommentId=15834503&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15834503

            People

            • Assignee:
              kkl0u Kostas Kloudas
              Reporter:
              rmetzger Robert Metzger
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development