Kafka
  1. Kafka
  2. KAFKA-469

Message size is not checked at the server

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.1
    • Fix Version/s: 0.7.2
    • Component/s: None
    • Labels:

      Description

      Message size is checked currently only in SyncProducer and not at the server. Therefore, non-java clients can push bigger messages to the server. Need a message size check at the server as well.

      1. kafka-469-v3.patch
        2 kB
        Swapnil Ghike
      2. kafka-469-v2.patch
        8 kB
        Swapnil Ghike
      3. kafka-469-v1.patch
        11 kB
        Swapnil Ghike

        Activity

        Hide
        Swapnil Ghike added a comment -

        Implemented message size check in Log.append(). The default maxMessageSize in server config is set to the same value as maxMessageSize in producer config.

        Will need to rebase after kafka-475 patch is checked in.

        Show
        Swapnil Ghike added a comment - Implemented message size check in Log.append(). The default maxMessageSize in server config is set to the same value as maxMessageSize in producer config. Will need to rebase after kafka-475 patch is checked in.
        Hide
        Jun Rao added a comment -

        Thanks for the patch. Looks good overall. One minor comment:

        Instead of passing in maxMessageSize in each log.append, is it better to pass it in the constructor of Log?

        Show
        Jun Rao added a comment - Thanks for the patch. Looks good overall. One minor comment: Instead of passing in maxMessageSize in each log.append, is it better to pass it in the constructor of Log?
        Hide
        Swapnil Ghike added a comment -

        Yes. Attached the change.

        Show
        Swapnil Ghike added a comment - Yes. Attached the change.
        Hide
        Neha Narkhede added a comment -

        +1 on v2. Thanks for the patch !

        Show
        Neha Narkhede added a comment - +1 on v2. Thanks for the patch !
        Hide
        Jun Rao added a comment -

        Thanks for patch v2. Committed to trunk.

        Show
        Jun Rao added a comment - Thanks for patch v2. Committed to trunk.
        Hide
        Swapnil Ghike added a comment -

        Reopening this issue to remove the stack trace logged in case of receiving a message larger than the permissible limit set by the server config.

        Show
        Swapnil Ghike added a comment - Reopening this issue to remove the stack trace logged in case of receiving a message larger than the permissible limit set by the server config.
        Hide
        Jun Rao added a comment -

        Thanks for patch v3. We need to log the topic/partition in handleProducerRequest() when we hit a MessageSizeTooLargeException. Committed to trunk with the fix.

        Show
        Jun Rao added a comment - Thanks for patch v3. We need to log the topic/partition in handleProducerRequest() when we hit a MessageSizeTooLargeException. Committed to trunk with the fix.

          People

          • Assignee:
            Swapnil Ghike
            Reporter:
            Swapnil Ghike
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 72h
              72h
              Remaining:
              Remaining Estimate - 72h
              72h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development