Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5065

Resource leaks in case of lost checkpoint messages

    XMLWordPrintableJSON

Details

    Description

      In case of lost checkpoint messages (AbstractCheckpointMessage) there won't be a clean up of the contained state handle done. This can happen in case of a JM crash if the TM had just sent a checkpoint message to the JM. Upon recovery, the JM won't notice that there was a checkpoint message which it had to handle.

      One way to mitigate the problem is to let the TM expect a response from the JM and if the JM failed to respond in time, to discard the state handle of the checkpoint message. Moreover, the JM could register in a fault tolerant store that it started a pending checkpoint. The entry should contain enough information (e.g. path for FileSystemStateBackend) to clean up a pending checkpoint with potentially lost checkpoint messages upon recovery of the JM.

      Attachments

        Activity

          People

            Unassigned Unassigned
            trohrmann Till Rohrmann
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: