Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3941

Avoid applying eviction listener in InMemoryKeyValueLoggedStore

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.1
    • Component/s: streams
    • Labels:
      None

      Description

      This is reported by dan norwood.

      In InMemoryKeyValueLoggedStore we set the eviction listener while creating the store, which records the evicted records as "deleted" in the changelogger, which will then send a tombstone record to the corresponding changelog topic partition. However, when restoring the store, although we are using the inner store's putInternal call and hence by-pass the logging since it is not needed, this eviction listener will still call the outer store's deleted call and hence still sends the tombstone record, causing the restoration process to fail, as it is not expecting the changelog log-end-offset to increase (i.e. more messages are appended to it) while restoration is going on.

      We should defer the listener initialization until the end of the init call after the restoration is completed, and also making sure the "register" call is made at the inner stores only.

        Attachments

          Activity

            People

            • Assignee:
              guozhang Guozhang Wang
              Reporter:
              guozhang Guozhang Wang

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment