Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-25919 Sink V2 improvements and followups
  3. FLINK-25857

Add committer metrics to track the status of committables

    XMLWordPrintableJSON

Details

    • Hide
      The TwoPhaseCommittingSink.createCommitter method parametrization has been changed, a new CommitterInitContext parameter has been added.

      The original method will remain available during the 1.19 release line, but they will be removed in consecutive releases.

      When migrating please also consider changes introduced by FLINK-33973 and FLIP-372 (https://cwiki.apache.org/confluence/display/FLINK/FLIP-372%3A+Enhance+and+synchronize+Sink+API+to+match+the+Source+API)
      Show
      The TwoPhaseCommittingSink.createCommitter method parametrization has been changed, a new CommitterInitContext parameter has been added. The original method will remain available during the 1.19 release line, but they will be removed in consecutive releases. When migrating please also consider changes introduced by FLINK-33973 and FLIP-372 ( https://cwiki.apache.org/confluence/display/FLINK/FLIP-372%3A+Enhance+and+synchronize+Sink+API+to+match+the+Source+API )

    Description

      With Sink V2 we can now track the progress of a committable during committing and show metrics about the committing status. (i.e. failed, retried, succeeded).

      The voted FLIP https://cwiki.apache.org/confluence/display/FLINK/FLIP-371%3A+Provide+initialization+context+for+Committer+creation+in+TwoPhaseCommittingSink

      Attachments

        1. screenshot-1.png
          258 kB
          wenchao.wu
        2. image-2023-10-20-17-23-09-595.png
          44 kB
          wenchao.wu

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. FLINK-33568 SinkV2MetricsITCase.testCommitterMetrics fails with NullPointerException

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. FLINK-33635 Some connectors can not compile in 1.19-SNAPSHOT

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          links to

          Web Link GitHub Pull Request #23555

          Web Link GitHub Pull Request #23876

          Activity

            People

              pvary Peter Vary
              fpaul Fabian Paul
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: