Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4741

Memory leak in RecordAccumulator.append

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0.0
    • Component/s: clients
    • Labels:
      None

      Description

      RecordAccumulator creates a `ByteBuffer` from free memory pool. This should be deallocated when invocations encounter an exception or throwing any exceptions.
      I added todo comment lines in the below code for cases to deallocate that buffer.

      RecordProducer.java
                  ByteBuffer buffer = free.allocate(size, maxTimeToBlock);
                  synchronized (dq) {
                      // Need to check if producer is closed again after grabbing the dequeue lock.
                      if (closed)
                         // todo buffer should be cleared.
                          throw new IllegalStateException("Cannot send after the producer is closed.");
      
                      // todo buffer should be cleared up when tryAppend throws an Exception
                      RecordAppendResult appendResult = tryAppend(timestamp, key, value, callback, dq);
                      if (appendResult != null) {
                          // Somebody else found us a batch, return the one we waited for! Hopefully this doesn't happen often...
                          free.deallocate(buffer);
                          return appendResult;
                      }
      
      

      I will raise PR for the same soon.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                satish.duggana Satish Duggana
                Reporter:
                satish.duggana Satish Duggana
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: