Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5467

Stateless chained tasks set legacy operator state

    Details

      Description

      I discovered this while trying to rescale a job with a Kafka source with a chained stateless operator.

      Looking into it, it turns out that this fails, because the checkpointed state contains legacy operator state for the chained operator although it is state less.

      /cc Aljoscha Krettek You mentioned that this might be a possible duplicate?

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3102

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3102
          Hide
          uce Ufuk Celebi added a comment -

          Fixed in
          af244aa (release-1.2)
          51a3573 (master).

          Show
          uce Ufuk Celebi added a comment - Fixed in af244aa (release-1.2) 51a3573 (master).
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on the issue:

          https://github.com/apache/flink/pull/3102

          Thanks! I verified that the adjusted test fails before this fix and runs fine after the fix. I'm going to merge this to master and release-1.2.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on the issue: https://github.com/apache/flink/pull/3102 Thanks! I verified that the adjusted test fails before this fix and runs fine after the fix. I'm going to merge this to master and release-1.2.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user StefanRRichter commented on the issue:

          https://github.com/apache/flink/pull/3102

          I modified a test to catch this problem, as suggested.

          Show
          githubbot ASF GitHub Bot added a comment - Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3102 I modified a test to catch this problem, as suggested.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on the issue:

          https://github.com/apache/flink/pull/3102

          I'm not aware of all implications of this change, but this fixes the problem I had. The operator does not checkpoint any legacy operator state.

          +1 to merge from my side.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on the issue: https://github.com/apache/flink/pull/3102 I'm not aware of all implications of this change, but this fixes the problem I had. The operator does not checkpoint any legacy operator state. +1 to merge from my side.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user StefanRRichter commented on the issue:

          https://github.com/apache/flink/pull/3102

          cc @uce @aljoscha

          Show
          githubbot ASF GitHub Bot added a comment - Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/3102 cc @uce @aljoscha
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user StefanRRichter opened a pull request:

          https://github.com/apache/flink/pull/3102

          FLINK-5467 Avoid legacy state for CheckpointedRestoring operators

          This PR fixes FLINK-5467.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/StefanRRichter/flink FixCheckointedRestoring

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3102.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3102


          commit efcb07198caafe4f28688775ea00e1048d08d532
          Author: Stefan Richter <s.richter@data-artisans.com>
          Date: 2017-01-12T11:24:34Z

          FLINK-5467 Avoid legacy state for CheckpointedRestoring operators


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user StefanRRichter opened a pull request: https://github.com/apache/flink/pull/3102 FLINK-5467 Avoid legacy state for CheckpointedRestoring operators This PR fixes FLINK-5467 . You can merge this pull request into a Git repository by running: $ git pull https://github.com/StefanRRichter/flink FixCheckointedRestoring Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3102.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3102 commit efcb07198caafe4f28688775ea00e1048d08d532 Author: Stefan Richter <s.richter@data-artisans.com> Date: 2017-01-12T11:24:34Z FLINK-5467 Avoid legacy state for CheckpointedRestoring operators

            People

            • Assignee:
              srichter Stefan Richter
              Reporter:
              uce Ufuk Celebi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development