Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-741

Improve log cleaning dedupe buffer efficiency

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 0.8.1
    • Component/s: None
    • Labels:
      None

      Description

      Two good suggestions:
      1. Use a probing scheme to increase density without increasing the collision rate
      2. Only count unique updates to the offset map (i.e. if the key is all zero, don't count it) when computing the load. Dynamically choose the end offset based on when the map is full.

      Would be good to investigate these things.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jkreps Jay Kreps
                Reporter:
                jkreps Jay Kreps
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: