Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-2713

Custom StateCheckpointers should be included in the snapshots

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      Currently the restoreInitialState call fails when the user uses a custom StateCheckpointer to create the snapshot, because the state is restored before the StateCheckpointer is set for the StreamOperatorState. (because the restoreInitialState() call precedes the open() call)

      To avoid this issue, the custom StateCheckpointer instance should be stored within the snapshot and should be set in the StreamOperatorState before calling restoreState(..).

      To reduce the overhead induced by this we can do 2 optimizations:

      • We only include custom StateCheckpointers (the default java serializer one is always available)
      • We only serialize the checkpointer once and store the byte array in the snapshot

      Attachments

        Activity

          People

            gyfora Gyula Fora
            gyfora Gyula Fora
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: