Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6853

Migrating from Flink 1.1 fails for FlinkCEP

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.3.1, 1.4.0
    • Component/s: CEP
    • Labels:
      None

      Description

      Migrating from Flink 1.1 fails for CEP, since in 1.1, the legacy MultiplexingStreamRecordSerializer is used for stream elements in the serialized priority queue (via the PriorityQueueSerializer).

      In newer versions, the StreamElementSerializer is used instead. For this to work, we need to implement the compatibility methods for StreamElementSerializer such that it is also compatible with configuration snapshots taken from the MultiplexingStreamRecordSerializer.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user tzulitai opened a pull request:

          https://github.com/apache/flink/pull/4079

          FLINK-6853 [DataStream] Let StreamRecordSerializer be compatible with MultiplexingStreamRecordSerializer

          This PR lets `StreamRecordSerializer.ensureCompatibility` be tolerable for config snapshots taken from the legacy `MultiplexingStreamRecordSerializer`. This is required for users which originally used `MultiplexingStreamRecordSerializer` to serialize stream elements as part of their checkpointed state (e.g. FlinkCEP).

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/tzulitai/flink FLINK-6853

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/4079.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #4079


          commit 54f5a33f8fb88d2219f06b6b874921d1d063ddca
          Author: Tzu-Li (Gordon) Tai <tzulitai@apache.org>
          Date: 2017-06-06T08:41:03Z

          FLINK-6853 [DataStream] Let StreamRecordSerializer be compatible with MultiplexingStreamRecordSerializer

          This commit lets StreamRecordSerializer.ensureCompatibility be tolerable
          for config snapshots taken from the legacy
          MultiplexingStreamRecordSerializer. This is required for users which
          originally used MultiplexingStreamRecordSerializer to serialize stream
          elements as part of their checkpointed state (e.g. FlnkCEP).


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user tzulitai opened a pull request: https://github.com/apache/flink/pull/4079 FLINK-6853 [DataStream] Let StreamRecordSerializer be compatible with MultiplexingStreamRecordSerializer This PR lets `StreamRecordSerializer.ensureCompatibility` be tolerable for config snapshots taken from the legacy `MultiplexingStreamRecordSerializer`. This is required for users which originally used `MultiplexingStreamRecordSerializer` to serialize stream elements as part of their checkpointed state (e.g. FlinkCEP). You can merge this pull request into a Git repository by running: $ git pull https://github.com/tzulitai/flink FLINK-6853 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4079.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4079 commit 54f5a33f8fb88d2219f06b6b874921d1d063ddca Author: Tzu-Li (Gordon) Tai <tzulitai@apache.org> Date: 2017-06-06T08:41:03Z FLINK-6853 [DataStream] Let StreamRecordSerializer be compatible with MultiplexingStreamRecordSerializer This commit lets StreamRecordSerializer.ensureCompatibility be tolerable for config snapshots taken from the legacy MultiplexingStreamRecordSerializer. This is required for users which originally used MultiplexingStreamRecordSerializer to serialize stream elements as part of their checkpointed state (e.g. FlnkCEP).
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tzulitai commented on the issue:

          https://github.com/apache/flink/pull/4079

          The test results in #4073 covers this change and results are green.
          Since #4073 is not really a real blocker for 1.3.1, while this fix is to allow CEP migration from 1.1, I'll merge this independent of #4073 first.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/4079 The test results in #4073 covers this change and results are green. Since #4073 is not really a real blocker for 1.3.1, while this fix is to allow CEP migration from 1.1, I'll merge this independent of #4073 first.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/4079

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4079
          Hide
          tzulitai Tzu-Li (Gordon) Tai added a comment -

          Fixed for master via 4895472ba2279d2982a45279d7be76bf3dfd8768.
          Fixed for 1.3.1 via 1d89dd06c1f9b09420ad3ff095d0842b4a951938.

          Show
          tzulitai Tzu-Li (Gordon) Tai added a comment - Fixed for master via 4895472ba2279d2982a45279d7be76bf3dfd8768. Fixed for 1.3.1 via 1d89dd06c1f9b09420ad3ff095d0842b4a951938.

            People

            • Assignee:
              tzulitai Tzu-Li (Gordon) Tai
              Reporter:
              tzulitai Tzu-Li (Gordon) Tai
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development