[SAMZA-762] java arrays not usable as keys anymore

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: kafka
• Assignee: József Márton Jung
• Reporter: Steven Aerts
• Created: 21/Aug/15 13:27
• Updated: 04/Feb/16 18:47

Description

Since the integration of the new kafka producer into samza, the partitioner is defined like this:

return abs(envelope.getPartitionKey.hashCode()) % numPartitions

This is problematic for java arrays, as their hash-code is independent of their content (unlike in scala). So a specific array based key is randomly assigned to a partition.

In 0.8 it was possible to configure your partitioner with systems.kafka.producer.partitioner.class and select for example the default murmur2 based partitioner of kafka.
But this is also not possible anymore.

Issue Links

• Is contained by: SAMZA-839 KafkaSystemProducer should use the same partitioning hash function as Kafka's producer