Kafka
  1. Kafka
  2. KAFKA-469

Message size is not checked at the server

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.1
    • Fix Version/s: 0.7.2
    • Component/s: None
    • Labels:

      Description

      Message size is checked currently only in SyncProducer and not at the server. Therefore, non-java clients can push bigger messages to the server. Need a message size check at the server as well.

      1. kafka-469-v3.patch
        2 kB
        Swapnil Ghike
      2. kafka-469-v2.patch
        8 kB
        Swapnil Ghike
      3. kafka-469-v1.patch
        11 kB
        Swapnil Ghike

        Activity

        Swapnil Ghike created issue -
        Swapnil Ghike made changes -
        Field Original Value New Value
        Status Open [ 1 ] In Progress [ 3 ]
        Jun Rao made changes -
        Fix Version/s 0.7.2 [ 12322475 ]
        Fix Version/s 0.7.1 [ 12319140 ]
        Hide
        Swapnil Ghike added a comment -

        Implemented message size check in Log.append(). The default maxMessageSize in server config is set to the same value as maxMessageSize in producer config.

        Will need to rebase after kafka-475 patch is checked in.

        Show
        Swapnil Ghike added a comment - Implemented message size check in Log.append(). The default maxMessageSize in server config is set to the same value as maxMessageSize in producer config. Will need to rebase after kafka-475 patch is checked in.
        Swapnil Ghike made changes -
        Attachment kafka-469-v1.patch [ 12542067 ]
        Swapnil Ghike made changes -
        Status In Progress [ 3 ] Patch Available [ 10002 ]
        Hide
        Jun Rao added a comment -

        Thanks for the patch. Looks good overall. One minor comment:

        Instead of passing in maxMessageSize in each log.append, is it better to pass it in the constructor of Log?

        Show
        Jun Rao added a comment - Thanks for the patch. Looks good overall. One minor comment: Instead of passing in maxMessageSize in each log.append, is it better to pass it in the constructor of Log?
        Hide
        Swapnil Ghike added a comment -

        Yes. Attached the change.

        Show
        Swapnil Ghike added a comment - Yes. Attached the change.
        Swapnil Ghike made changes -
        Attachment kafka-469-v2.patch [ 12542207 ]
        Hide
        Neha Narkhede added a comment -

        +1 on v2. Thanks for the patch !

        Show
        Neha Narkhede added a comment - +1 on v2. Thanks for the patch !
        Hide
        Jun Rao added a comment -

        Thanks for patch v2. Committed to trunk.

        Show
        Jun Rao added a comment - Thanks for patch v2. Committed to trunk.
        Jun Rao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Jun Rao made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Swapnil Ghike added a comment -

        Reopening this issue to remove the stack trace logged in case of receiving a message larger than the permissible limit set by the server config.

        Show
        Swapnil Ghike added a comment - Reopening this issue to remove the stack trace logged in case of receiving a message larger than the permissible limit set by the server config.
        Swapnil Ghike made changes -
        Attachment kafka-469-v3.patch [ 12542885 ]
        Hide
        Jun Rao added a comment -

        Thanks for patch v3. We need to log the topic/partition in handleProducerRequest() when we hit a MessageSizeTooLargeException. Committed to trunk with the fix.

        Show
        Jun Rao added a comment - Thanks for patch v3. We need to log the topic/partition in handleProducerRequest() when we hit a MessageSizeTooLargeException. Committed to trunk with the fix.
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail [ 12721755 ] Apache Kafka Workflow [ 13052740 ]
        Tony Stevenson made changes -
        Workflow Apache Kafka Workflow [ 13052740 ] no-reopen-closed, patch-avail [ 13055380 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        31s 1 Swapnil Ghike 18/Aug/12 01:46
        In Progress In Progress Patch Available Patch Available
        5d 10m 1 Swapnil Ghike 23/Aug/12 01:57
        Patch Available Patch Available Resolved Resolved
        1d 13h 36m 1 Jun Rao 24/Aug/12 15:33
        Resolved Resolved Closed Closed
        4s 1 Jun Rao 24/Aug/12 15:33

          People

          • Assignee:
            Swapnil Ghike
            Reporter:
            Swapnil Ghike
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 72h
              72h
              Remaining:
              Remaining Estimate - 72h
              72h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development