Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5145

Remove task close() call from closeNonAssignedSuspendedTasks method

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 0.10.2.0
    • None
    • streams

    Description

      While rebalancing ProcessorNode.close() can be called twice, once from StreamThread.suspendTasksAndState() and once from StreamThread.closeNonAssignedSuspendedTasks(). If ProcessorNode.close() throws some exception because of calling close() multiple times( i.e. IllegalStateException from some KafkaConsumer instance being used by some processor for some lookup), we fail to close the task's state manager ( i.e. call to task.closeStateManager(true); fails). After rebalance, if the same task id is launched on same application instance but in different thread then the task get stuck because it fails to get lock to the task's state directory.

      Since processor close() is already called from StreamThread.suspendTasksAndState() we don't need to call again from StreamThread.closeNonAssignedSuspendedTasks().

      Attachments

        1. logs.txt
          105 kB
          Narendra Kumar
        2. DebugTransformer.java
          2 kB
          Narendra Kumar
        3. BugTest.java
          2 kB
          Narendra Kumar

        Issue Links

          Activity

            It seems like ProcessorNode.close() method isn't actually called from StreamThread.suspendTasksAndState().

            evis Evgeny Veretennikov added a comment - It seems like ProcessorNode.close() method isn't actually called from StreamThread.suspendTasksAndState().

            This seems to be a duplicate: https://issues.apache.org/jira/browse/KAFKA-5167

            Narendra Kumar Can you confirm?

            mjsax Matthias J. Sax added a comment - This seems to be a duplicate: https://issues.apache.org/jira/browse/KAFKA-5167 Narendra Kumar Can you confirm?
            mingleizhang zhangminglei added a comment -

            I am a new to kafka. Can anyone give me a permission that contribute to Kafka ?

            mingleizhang zhangminglei added a comment - I am a new to kafka. Can anyone give me a permission that contribute to Kafka ?

            mingleizhang Thanks for starting to contribute to Kafka! I just realized, that this is a duplicate and there is already a PR for the other Jira (cf KAFKA-5485). There should be other "newbie" Jira thought you can pick up.

            damianguy Can you add mingleizhang to the contributor list, so he can assign Jiras to himself. Thx.

            mjsax Matthias J. Sax added a comment - mingleizhang Thanks for starting to contribute to Kafka! I just realized, that this is a duplicate and there is already a PR for the other Jira (cf KAFKA-5485 ). There should be other "newbie" Jira thought you can pick up. damianguy Can you add mingleizhang to the contributor list, so he can assign Jiras to himself. Thx.

            People

              Unassigned Unassigned
              Narendra Kumar Narendra Kumar
              Guozhang Wang Guozhang Wang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: