Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7830

Convert Kafka RPCs to use automatically generated code

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • None
    • None
    • clients, core
    • None

    Description

      KAFKA-7609 added a way of automatically generating code for reading and writing Kafka RPC message types from JSON schemas.

      Automatically generated code is preferrable to manually written serialization code.

      • * It is less tedious and error-prone to use than hand-written code.
      • For developers writing Kafka clients in other languages, the JSON schemas are useful in a way that the java serialization code is not.
      • It will eventually be possible to automatically validate aspects of cross-version compatibility, when using JSON message schemas.
      • Once all of the RPCs are converted, we can drop using Structs in favor of serializing directly to ByteBuffer, to reduce GC load.

      This JIRA tracks converting the current hand-written message serialization code to automatically generated serialization.

      Attachments

        1.
        		 Add Protocol Generator for Kafka Sub-task Resolved Colin McCabe
        2.
        		 Use automatic RPC generation in CreateTopics Sub-task Resolved Colin McCabe
        3.
        		 Support null defaults in KAFKA-7609 RPC specifications Sub-task Resolved Colin McCabe
        4.
        		 Add error specifications to KAFKA-7609 Sub-task Patch Available Tom Bentley
        5.
        		 Replace JoinGroup request/response with automated protocol Sub-task Resolved Boyang Chen
        6.
        		 Replace LeaveGroup request/response with automated protocol Sub-task Resolved Boyang Chen
        7.
        		 Replace OffsetCommit request/response with automated protocol Sub-task Resolved Boyang Chen
        8.
        		 Replace SaslHandshake request/response with automated protocol Sub-task Resolved Mickael Maison
        9.
        		 Replace SaslAuthenticate request/response with automated protocol Sub-task Resolved Mickael Maison
        10.
        		 Replace DeleteTopics request/response with automated protocol Sub-task Resolved Mickael Maison
        11.
        		 Replace FindCoordinator request/response with automated protocol Sub-task Resolved Mickael Maison
        12.
        		 Replace ControlledShutdown request/response with automated protocol Sub-task Resolved Mickael Maison
        13.
        		 Replace InitProducerId request/response with automated protocol Sub-task Resolved Jason Gustafson
        14.
        		 Replace Heartbeat request/response with automated protocol Sub-task Resolved Mickael Maison
        15.
        		 Replace SyncGroup request/response with automated protocol Sub-task Resolved Boyang Chen
        16.
        		 Replace CreateDelegationToken request/response with automated protocol Sub-task Resolved Mickael Maison
        17.
        		 Replace ListGroups request/response with automated protocol Sub-task Resolved Boyang Chen
        18.
        		 Replace DeleteGroups request/response with automated protocol Sub-task Resolved Boyang Chen
        19.
        		 Replace AddOffsetsToTxn request/response with automated protocol Sub-task Resolved Boyang Chen
        20.
        		 Replace control requests/responses with automated protocol Sub-task Resolved Ismael Juma
        21.
        		 Allow "bytes" type to generated a ByteBuffer rather than byte arrays Sub-task Resolved Nikolay Izhikov
        22.
        		 Replace TxnOffsetCommitRequest request/response with automated protocol Sub-task Resolved Boyang Chen
        23.
        		 Replace RenewDelegationToken request/response with automated protocol Sub-task Resolved Mickael Maison
        24.
        		 Replace ExpireDelegationToken request/response with automated protocol Sub-task Resolved Mickael Maison
        25.
        		 Replace DescribeDelegationToken request/response with automated protocol Sub-task Resolved Viktor Somogyi-Vass
        26.
        		 Replace ApiVersionsRequest request/response with automated protocol Sub-task Resolved Boyang Chen
        27.
        		 Replace EndTxn request/response with automated protocol Sub-task Resolved Boyang Chen
        28.
        		 Replace WriteTxnMarkers request/response with automated protocol Sub-task Resolved Boyang Chen
        29.
        		 Replace AddPartitionsToTxn request/response with automated protocol Sub-task Resolved Boyang Chen
        30.
        		 Replace OffsetFetch request/response with automated protocol Sub-task Resolved Boyang Chen
        31.
        		 Replace DeleteRecords request/response with automated protocol Sub-task Resolved Tom Bentley
        32.
        		 Replace CreatePartitions request/response with automated protocol Sub-task Resolved Vikas Singh
        33.
        		 Replace DescribeAcls request/response with automated protocol Sub-task Resolved Mickael Maison
        34.
        		 Replace CreateAcls request/response with automated protocol Sub-task Resolved Ismael Juma
        35.
        		 Replace DeleteAcls request/response with automated protocol Sub-task Resolved Ismael Juma
        36.
        		 Replace DescribeConfigs request/response with automated protocol Sub-task Resolved Tom Bentley
        37.
        		 Replace AlterConfigs request/response with automated protocol Sub-task Resolved Tom Bentley
        38.
        		 Replace AlterReplicaLogDirs request/response with automated protocol Sub-task Resolved Tom Bentley
        39.
        		 Replace DescribeLogDirs request/response with automated protocol Sub-task Resolved Tom Bentley
        40.
        		 Replace ListOffset request/response with automated protocol Sub-task Resolved Mickael Maison
        41.
        		 Replace Produce request/response with automated protocol Sub-task Resolved Chia-Ping Tsai
        42.
        		 Replace Fetch request/response with automated protocol Sub-task Resolved David Arthur
        43.
        		 Replace OffsetsForLeaderEpoch request/response with automated protocol Sub-task Resolved David Jacot

        Activity

          People

            cmccabe Colin McCabe
            cmccabe Colin McCabe
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: