Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1958

ZookeeperConsumerConnector doesn't remove consumer node on shutdown.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Auto Closed
    • Affects Version/s: 0.8.1.1
    • Fix Version/s: None
    • Component/s: consumer

      Description

      We use kafka with ZooKeeper via high level consumer.
      There is a scheduled job that creates a consumer with specific group, does necessary logic and shuts down this consumer.

      An issue:
      Nobody deletes /consumers/myGroup/ids/myGroup_<ip>_<postfix>. And after several job runs there are a lot of dead consumer IDs under myGroup. I've got an issue that new consumer doesn't see a partition.

      We start to implement an approach to remove a consumer nodes from Zookeeper manually after consumer is shutted down.
      I think better way to remove this node during ZookeeperConsumerConnector.shutdown().

      P.S.:
      If I missed something in your sources please let me know.

        Attachments

          Activity

            People

            • Assignee:
              nehanarkhede Neha Narkhede
              Reporter:
              beletsky.andrey Beletsky Andrey
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: