Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3563

Maintain MessageAndMetadata constructor compatibility

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      The MessageAndMetadata constructor was changed to include timestamp information as a part of KIP-32. Though the constructor may not be used in general client usage, it may be used in unit tests or some advanced usage. We should maintain compatibility if possible.

      One example where the constructor is used is Apache Spark: https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala#L223-L225

      The old constructor was:

      MessageAndMetadata[K, V](topic: String,
         partition: Int,
         private val rawMessage: Message,
         offset: Long,
         keyDecoder: Decoder[K], valueDecoder: Decoder[V])
      

      And after KIP-32 it is now:

      MessageAndMetadata[K, V](topic: String,
         partition: Int,
         private val rawMessage: Message,
         offset: Long,
         timestamp: Long = Message.NoTimestamp,
         timestampType: TimestampType = TimestampType.CREATE_TIME,
         keyDecoder: Decoder[K], valueDecoder: Decoder[V])
      

      Even though timestamp and timestampType have defaults, if keyDecoder and valueDecoder were not accessed by name, then the new constructor is not backwards compatible.

      We can fix compatibility by moving the timestamp and timestampType parameters to the end of the constructor, or by providing a new constructor without timestamp and timestampType that matches the old constructor.

        Attachments

          Activity

            People

            • Assignee:
              granthenke Grant Henke
              Reporter:
              granthenke Grant Henke
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: