Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1017

High number of open file handles in 0.8 producer

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: producer
    • Labels:
      None

      Description

      Reported by Jun Rao:

      For over-partitioned topics, each broker could be the leader for at least 1 partition. In the producer, we randomly select a partition to send the data. Pretty soon, each producer will establish a connection to each of the n brokers. Effectively, we increased the # of socket connections by a factor of n, compared to 0.7.

      The increased number of socket connections increases the number of open file handles, this could come pretty close to the OS limit.

      1. kafka-1017.patch
        8 kB
        Swapnil Ghike

        Activity

        Hide
        swapnilghike Swapnil Ghike added a comment -

        This patch fixes two issues:

        1. Reduce the number of open file handles on the producer: I think we can clear the topic-partition send-cache on the EventHandler once in every refresh metadata interval. This will make the producer produce to the same partition for say, 15 mins, for a given topic. That should be ok. The topic-partition send-cache will be refreshed when the metadata is refreshed.

        2. There is a race condition in Producer.send() and Producer.close(). This can lead to reopening of a closed ProducerPool and thereby socket leaks.

        Show
        swapnilghike Swapnil Ghike added a comment - This patch fixes two issues: 1. Reduce the number of open file handles on the producer: I think we can clear the topic-partition send-cache on the EventHandler once in every refresh metadata interval. This will make the producer produce to the same partition for say, 15 mins, for a given topic. That should be ok. The topic-partition send-cache will be refreshed when the metadata is refreshed. 2. There is a race condition in Producer.send() and Producer.close(). This can lead to reopening of a closed ProducerPool and thereby socket leaks.
        Hide
        nehanarkhede Neha Narkhede added a comment -

        Thanks for fixing this issue, Swapnil. The patch looks good to me. +1

        Show
        nehanarkhede Neha Narkhede added a comment - Thanks for fixing this issue, Swapnil. The patch looks good to me. +1
        Hide
        junrao Jun Rao added a comment -

        Good patch. +1. Committed to 0.8.

        Show
        junrao Jun Rao added a comment - Good patch. +1. Committed to 0.8.
        Hide
        guozhang Guozhang Wang added a comment -

        Good catch, thanks Swapnil!

        Show
        guozhang Guozhang Wang added a comment - Good catch, thanks Swapnil!

          People

          • Assignee:
            swapnilghike Swapnil Ghike
            Reporter:
            swapnilghike Swapnil Ghike
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development