Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20654

Unaligned checkpoint recovery may lead to corrupted data stream

    XMLWordPrintableJSON

Details

    Description

      Fix of FLINK-20433 shows potential corruption after recovery for all variations of UnalignedCheckpointITCase.

      To reproduce, run UCITCase a couple hundreds times. The issue showed for me in:

      • execute [Parallel union, p = 5]
      • execute [Parallel union, p = 10]
      • execute [Parallel cogroup, p = 5]
      • execute [parallel pipeline with remote channels, p = 5]
        with decreasing frequency.

      The issue manifests as one of the following issues:

      • stream corrupted exception
      • EOF exception
      • assertion failure in NUM_LOST or NUM_OUT_OF_ORDER
      • (for union) ArithmeticException overflow (because the number that should be [0;100000] has been mis-deserialized)

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. FLINK-21104 UnalignedCheckpointITCase.execute failed with "IllegalStateException"

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-20309 UnalignedCheckpointTestBase.execute is failed

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. FLINK-20662 UnalignedCheckpointITCase.execute failed with IndexOutOfBoundsException

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-20744 org.apache.flink.test.checkpointing.UnalignedCheckpointITCase fails due to java.lang.ArrayIndexOutOfBoundsException

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-20960 Add warning in 1.12 release notes about potential corrupt data stream with unaligned checkpoint

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #14445

          Web Link GitHub Pull Request #14471

          Web Link GitHub Pull Request #14509

          Web Link GitHub Pull Request #14525

          Web Link GitHub Pull Request #14581

          Web Link GitHub Pull Request #14582

          Web Link GitHub Pull Request #14728

          Web Link GitHub Pull Request #14736

          Web Link GitHub Pull Request #14797

          Web Link GitHub Pull Request #14807

          Web Link GitHub Pull Request #14817

          Web Link GitHub Pull Request #15252

          Web Link GitHub Pull Request #15714

          Web Link pull/14721

          Web Link pull/14730

          Activity

            People

              pnowojski Piotr Nowojski
              arvid Arvid Heise
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: