-
Type:
Improvement
-
Status: Resolved
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: 1.5.0.1
-
Fix Version/s: 1.6.0
-
Component/s: Sinks+Sources
-
Labels:None
I believe it would be useful to include a message's key in the event header, as users will often want to partition based on this key when ingesting into a sink.
Example:
sfdc.sinks.hdfs.hdfs.path = /data/testing/kafka_flume/dataset=%{key}/year=%Y/month=%m/day=%d/
In this case, the first part of the partition key is the Kafka message key, which will ensure messages are organized by key.
I also updated some of the comments and cleaned up the code slightly.
Cheers,
Ricky