Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3029

Make class org.apache.kafka.common.TopicPartition Serializable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: 0.9.0.1, 0.10.0.0
    • Component/s: clients
    • Labels:
    • Flags:
      Patch, Important

      Description

      The client class TopicPartition is exposed and used by consumer applications directly. In case where the application needs to checkpoint the state it is difficult to serialize different app classes that use TopicPartition as TopicParitition is not serializable.

      For instance consider the Spark use case where RDDs have to be checkpointed....the KafakaInputDstream (which we are currently modifying to use the new Kafka API rather than the Highlevel apis in previous version) cannot be serialized due to above limitation.

      I have created a patch to serialize TopicPartition class by making it implement serializable interface and have issued a pull request.

      Can this be merged for the next release (0.9.0.1)

      Thanks

      Praveen

        Attachments

          Activity

            People

            • Assignee:
              praveend Praveen Devarao
              Reporter:
              praveend Praveen Devarao
              Reviewer:
              Ewen Cheslack-Postava
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: