Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1958

ZookeeperConsumerConnector doesn't remove consumer node on shutdown.

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Auto Closed
    • 0.8.1.1
    • None
    • consumer

    Description

      We use kafka with ZooKeeper via high level consumer.
      There is a scheduled job that creates a consumer with specific group, does necessary logic and shuts down this consumer.

      An issue:
      Nobody deletes /consumers/myGroup/ids/myGroup_<ip>_<postfix>. And after several job runs there are a lot of dead consumer IDs under myGroup. I've got an issue that new consumer doesn't see a partition.

      We start to implement an approach to remove a consumer nodes from Zookeeper manually after consumer is shutted down.
      I think better way to remove this node during ZookeeperConsumerConnector.shutdown().

      P.S.:
      If I missed something in your sources please let me know.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            nehanarkhede Neha Narkhede
            beletsky.andrey Beletsky Andrey
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment