Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3941

Avoid applying eviction listener in InMemoryKeyValueLoggedStore

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.1
    • Component/s: streams
    • Labels:
      None

      Description

      This is reported by dan norwood.

      In InMemoryKeyValueLoggedStore we set the eviction listener while creating the store, which records the evicted records as "deleted" in the changelogger, which will then send a tombstone record to the corresponding changelog topic partition. However, when restoring the store, although we are using the inner store's putInternal call and hence by-pass the logging since it is not needed, this eviction listener will still call the outer store's deleted call and hence still sends the tombstone record, causing the restoration process to fail, as it is not expecting the changelog log-end-offset to increase (i.e. more messages are appended to it) while restoration is going on.

      We should defer the listener initialization until the end of the init call after the restoration is completed, and also making sure the "register" call is made at the inner stores only.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                guozhang Guozhang Wang
                Reporter:
                guozhang Guozhang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: