Kafka
  1. Kafka
  2. KAFKA-1092

Add server config parameter to separate bind address and ZK hostname

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.1
    • Fix Version/s: 0.8.1
    • Component/s: config
    • Labels:
      None

      Description

      Currently, in server.properties, you can configure host.name which gets used for two purposes: 1) to bind the socket 2) to publish the broker details to ZK for clients to use.

      There are times when these two settings need to be different. Here's an example. I want to setup Kafka brokers on OpenStack virtual machines in a private cloud but I need producers to connect from elsewhere on the internal corporate network. With OpenStack, the virtual machines are only exposed to DHCP addresses (typically RFC 1918 private addresses). You can assign "floating ips" to a virtual machine but it's forwarded using Network Address Translation and not exposed directly to the VM. Also, there's typically no DNS to provide hostname lookup. Hosts have names like "fubar.novalocal" that are not externally routable.

      Here's what I want. I want the broker to bind to the VM's private network IP but I want it to publish it's floating IP to ZooKeeper so that producers can publish to it.

      I propose a new optional parameter, "listen", which would allow you to specify the socket address to listen on. If not set, the parameter would default to host.name, which is the current behavior.

      #Publish the externally routable IP in ZK
      host.name = <floating ip>
      #Accept connections from any interface the VM knows about
      listen = *

      1. KAFKA-1092.patch
        29 kB
        Roger Hoover
      2. KAFKA-1092.patch
        18 kB
        Roger Hoover

        Activity

        Roger Hoover created issue -
        Roger Hoover made changes -
        Field Original Value New Value
        Attachment KAFKA-1092.patch [ 12610686 ]
        Roger Hoover made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Roger Hoover made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Roger Hoover made changes -
        Attachment KAFKA-1092.patch [ 12611182 ]
        Roger Hoover made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jun Rao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Assignee Roger Hoover [ theduderog ]
        Fix Version/s 0.8.1 [ 12322960 ]
        Resolution Fixed [ 1 ]
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail [ 12821532 ] Apache Kafka Workflow [ 13051191 ]
        Tony Stevenson made changes -
        Workflow Apache Kafka Workflow [ 13051191 ] no-reopen-closed, patch-avail [ 13053584 ]

          People

          • Assignee:
            Roger Hoover
            Reporter:
            Roger Hoover
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development