Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21178

Task failure will not trigger master hook's reset()

    XMLWordPrintableJSON

    Details

      Description

      In Pravega Flink connector integration with Flink 1.12, we found an issue with our no-checkpoint recovery test case [1].

      We expect the recovery will call the ReaderCheckpointHook::reset() function which was the behaviour before 1.12. However FLINK-20222 changes the logic, the reset() call will only be called along with a global recovery. This causes Pravega source data loss when failure happens before the first checkpoint.

      [1]  https://github.com/crazyzhou/flink-connectors/blob/da9f76d04404071471ebd86bf6889b307c9122ff/src/test/java/io/pravega/connectors/flink/FlinkPravegaReaderRGStateITCase.java#L78

        Attachments

          Activity

            People

            • Assignee:
              Brian Zhou Brian Zhou
              Reporter:
              Brian Zhou Brian Zhou
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: