Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3184

Add Checkpoint for In-memory State Store

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: streams
    • Labels:

      Description

      Currently Kafka Streams does not make a checkpoint of the persistent state store upon committing, which would be expensive since it is "stopping the world" and write on disks: for example, RocksDB would require you to copy the file directory to make a copy naively.

      However, for in-memory stores checkpointing maybe doable in an asynchronous manner hence it can be done quickly. And the benefit of having intermediate checkpoint is to avoid restoring from scratch if standby tasks are not present.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              guozhang Guozhang Wang
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: