Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4203

Java producer default max message size does not align with broker default

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.1
    • Fix Version/s: 2.5.0
    • Component/s: None
    • Labels:
      None

      Description

      The Java producer sets max.request.size = 1048576 (the base 2 version of 1 MB (MiB))

      The broker sets max.message.bytes = 1000012 (the base 10 value of 1 MB + 12 bytes for overhead)

      This means that by default the producer can try to produce messages larger than the broker will accept resulting in RecordTooLargeExceptions.

      There were not similar issues in the old producer because it sets max.message.size = 1000000 (the base 10 value of 1 MB)

      I propose we increase the broker default for max.message.bytes to 1048588 (the base 2 value of 1 MB (MiB) + 12 bytes for overhead) so that any message produced with default configs from either producer does not result in a RecordTooLargeException.

        Attachments

          Activity

            People

            • Assignee:
              ijuma Ismael Juma
              Reporter:
              granthenke Grant Henke
              Reviewer:
              Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: