Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4815 Idempotent/transactional Producer (KIP-98)
  3. KAFKA-5196

LogCleaner should be transaction-aware

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 0.11.0.0
    • clients, core, producer
    • None

    Description

      In accordance with the KIP-98 design, the log cleaner should be aware of transaction markers. This means the following:

      1. Cleaning should be restricted below the last stable offset (which ensures that only decided data is considered when cleaning).
      2. Records from aborted transactions are removed immediately.
      3. Removal of the COMMIT/ABORT markers must be delayed to ensure that they are not missed by concurrent readers.

      Attachments

        Issue Links

          fixes

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-6780 log cleaner shouldn't clean messages beyond high watermark

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #3008

          Activity

            People

              hachikuji Jason Gustafson
              hachikuji Jason Gustafson
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: