Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2176

DefaultPartitioner doesn't perform consistent hashing based on

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.8.1
    • Fix Version/s: 0.8.1
    • Component/s: core
    • Labels:
    • Flags:
      Patch

      Description

      While deploying MirrorMakers in production, we configured it to use kafka.producer.DefaultPartitioner. By doing this and since we had the same amount partitions for the topic in local and aggregation cluster, we expect that the messages will be partitioned the same way.

      This wasn't the case. Messages were properly partitioned with DefaultPartitioner on our local cluster, since the key was of the type String.
      On the MirrorMaker side, the messages were not properly partitioned.

      Problem is that the Array[Byte] doesn't implement hashCode function, since it is mutable collection.

      Fix is to calculate the deep hash code if the key is of Array type.

        Attachments

        1. KAFKA-2176.patch
          6 kB
          Igor Maravić

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              i_maravic Igor Maravić
              Reviewer:
              Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: