Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-7210

Add TwoPhaseCommitSinkFunction (implementing exactly-once semantic in generic way)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • Connectors / Common
    • None

    Description

      To implement exactly-once sink there is a re-occurring pattern for doing it - two phase commit algorithm. It is used both in `BucketingSink` and in `Pravega` sink and it will be used in `Kafka 0.11` connector. It would be good to extract this common logic into one class, both to improve existing implementation (for exampe `Pravega`'s sink doesn't abort interrupted transactions) and to make it easier for the users to implement their own custom exactly-once sinks.

      Attachments

        Issue Links

          Activity

            People

              pnowojski Piotr Nowojski
              pnowojski Piotr Nowojski
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: