Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7830

Convert Kafka RPCs to use automatically generated code

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: clients, core
    • Labels:
      None

      Description

      KAFKA-7609 added a way of automatically generating code for reading and writing Kafka RPC message types from JSON schemas.

      Automatically generated code is preferrable to manually written serialization code.

      • * It is less tedious and error-prone to use than hand-written code.
      • For developers writing Kafka clients in other languages, the JSON schemas are useful in a way that the java serialization code is not.
      • It will eventually be possible to automatically validate aspects of cross-version compatibility, when using JSON message schemas.
      • Once all of the RPCs are converted, we can drop using Structs in favor of serializing directly to ByteBuffer, to reduce GC load.

      This JIRA tracks converting the current hand-written message serialization code to automatically generated serialization.

        Attachments

        1.
        Add Protocol Generator for Kafka Sub-task Resolved Colin McCabe
        2.
        Use automatic RPC generation in CreateTopics Sub-task Resolved Colin McCabe
        3.
        Support null defaults in KAFKA-7609 RPC specifications Sub-task Resolved Colin McCabe
        4.
        Add error specifications to KAFKA-7609 Sub-task Open Tom Bentley
        5.
        Replace JoinGroup request/response with automated protocol Sub-task Resolved Boyang Chen
        6.
        Replace LeaveGroup request/response with automated protocol Sub-task Resolved Boyang Chen
        7.
        Replace OffsetCommit request/response with automated protocol Sub-task Resolved Boyang Chen
        8.
        Replace SaslHandshake request/response with automated protocol Sub-task Resolved Mickael Maison
        9.
        Replace SaslAuthenticate request/response with automated protocol Sub-task Resolved Mickael Maison
        10.
        Replace DeleteTopics request/response with automated protocol Sub-task Resolved Mickael Maison
        11.
        Replace FindCoordinator request/response with automated protocol Sub-task Resolved Mickael Maison
        12.
        Replace ControlledShutdown request/response with automated protocol Sub-task Resolved Mickael Maison
        13.
        Replace InitProducerId request/response with automated protocol Sub-task Resolved Jason Gustafson
        14.
        Replace Heartbeat request/response with automated protocol Sub-task Resolved Mickael Maison
        15.
        Replace SyncGroup request/response with automated protocol Sub-task Resolved Boyang Chen
        16.
        Replace CreateDelegationToken request/response with automated protocol Sub-task Resolved Mickael Maison
        17.
        Replace ListGroups request/response with automated protocol Sub-task Resolved Boyang Chen
        18.
        Replace DeleteGroups request/response with automated protocol Sub-task Resolved Boyang Chen
        19.
        Replace AddOffsetsToTxn request/response with automated protocol Sub-task Open Boyang Chen
        20.
        Replace control requests/responses with automated protocol Sub-task Resolved Ismael Juma
        21.
        Allow "bytes" type to generated a ByteBuffer rather than byte arrays Sub-task Resolved Nikolay Izhikov
        22.
        Replace TxnOffsetCommitRequest request/response with automated protocol Sub-task Resolved Boyang Chen
        23.
        Replace RenewDelegationToken request/response with automated protocol Sub-task Resolved Mickael Maison
        24.
        Replace ExpireDelegationToken request/response with automated protocol Sub-task Resolved Mickael Maison
        25.
        Replace DescribeDelegationToken request/response with automated protocol Sub-task Resolved Viktor Somogyi-Vass
        26.
        Replace ApiVersionsRequest request/response with automated protocol Sub-task Resolved Boyang Chen
        27.
        Replace EndTxn request/response with automated protocol Sub-task Resolved Boyang Chen
        28.
        Replace WriteTxnMarkers request/response with automated protocol Sub-task Open Boyang Chen
        29.
        Replace AddPartitionsToTxn request/response with automated protocol Sub-task Open Boyang Chen
        30.
        Replace OffsetFetch request/response with automated protocol Sub-task Resolved Boyang Chen
        31.
        Replace DeleteRecords request/response with automated protocol Sub-task Patch Available Tom Bentley
        32.
        Replace CreatePartitions request/response with automated protocol Sub-task Resolved Vikas Singh
        33.
        Replace DescribeAcls request/response with automated protocol Sub-task Resolved Mickael Maison
        34.
        Replace CreateAcls request/response with automated protocol Sub-task Resolved Ismael Juma
        35.
        Replace DeleteAcls request/response with automated protocol Sub-task Resolved Ismael Juma
        36.
        Replace DescribeConfigs request/response with automated protocol Sub-task Open Tom Bentley
        37.
        Replace AlterConfigs request/response with automated protocol Sub-task Open Tom Bentley
        38.
        Replace AlterReplicaLogDirs request/response with automated protocol Sub-task Open Tom Bentley
        39.
        Replace DescribeLogDirs request/response with automated protocol Sub-task Patch Available Tom Bentley

          Activity

            People

            • Assignee:
              cmccabe Colin McCabe
              Reporter:
              cmccabe Colin McCabe
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: