Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-762

java arrays not usable as keys anymore

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: kafka
    • Labels:
      None

      Description

      Since the integration of the new kafka producer into samza, the partitioner is defined like this:

      return abs(envelope.getPartitionKey.hashCode()) % numPartitions
      

      This is problematic for java arrays, as their hash-code is independent of their content (unlike in scala). So a specific array based key is randomly assigned to a partition.

      In 0.8 it was possible to configure your partitioner with systems.kafka.producer.partitioner.class and select for example the default murmur2 based partitioner of kafka.
      But this is also not possible anymore.

        Attachments

        1. SAMZA-762.0.patch
          22 kB
          József Márton Jung

          Issue Links

            Activity

              People

              • Assignee:
                jjung József Márton Jung
                Reporter:
                steven.aerts Steven Aerts
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: