Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9373

Improve shutdown performance via lazy accessing the offset and time indices.

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0, 2.4.0, 2.3.1
    • 2.6.0
    • log
    • None

    Description

      KAFKA-7283 enabled lazy mmap on index files by initializing indices on-demand rather than performing costly disk/memory operations when creating all indices on broker startup. This helped reducing the startup time of brokers. However, segment indices are still created on closing segments, regardless of whether they need to be closed or not.

      Ideally we should:

      • Improve shutdown performance via lazy accessing the offset and time indices.
      • Eliminate redundant disk accesses and memory mapped operations while deleting or renaming files that back segment indices.
      • Prevent illegal accesses to underlying indices of a closed segment, which would lead to memory leaks due to recreation of the underlying memory mapped objects.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            agencer Adem Efe Gencer
            agencer Adem Efe Gencer
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment