Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-741

Improve log cleaning dedupe buffer efficiency

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • 0.8.1
    • None
    • None

    Description

      Two good suggestions:
      1. Use a probing scheme to increase density without increasing the collision rate
      2. Only count unique updates to the offset map (i.e. if the key is all zero, don't count it) when computing the load. Dynamically choose the end offset based on when the map is full.

      Would be good to investigate these things.

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-739 Handle null values in Message payload

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              jkreps Jay Kreps
              jkreps Jay Kreps
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: