Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1.0
    • Fix Version/s: 0.10.1.0
    • Component/s: streams
    • Labels:
      None

      Description

      In ProcessorStateManager in the lockStateDirectory method, we close a channel if the lock is null. However, as described in https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html, "...closing a channel releases all locks held by the Java virtual machine on the underlying file regardless of whether the locks were acquired via that channel or via another channel open on the same file. It is strongly recommended that, within a program, a unique channel be used to acquire all locks on any given file.". Thus closing this channel sometimes leads to the other lock in ProcessorStateManager, called directoryLock to be released.

      Problem is reproduced as part of the steps in KAFKA-3805, when two processes are using the same state directory.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/kafka/pull/1641

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1641
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user dguy opened a pull request:

          https://github.com/apache/kafka/pull/1641

          HOTFIX: fix compilation error due to merge of KAFKA-3812

          Merge of KAFKA-3812 caused a compilation error in StreamThreadStateStoreProviderTest

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/dguy/kafka fix-compile-error

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/kafka/pull/1641.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1641


          commit 60fa103e3a6fe1e4dd9788033e854886493016e0
          Author: Damian Guy <damian.guy@gmail.com>
          Date: 2016-07-19T23:49:02Z

          fix compilation error


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/1641 HOTFIX: fix compilation error due to merge of KAFKA-3812 Merge of KAFKA-3812 caused a compilation error in StreamThreadStateStoreProviderTest You can merge this pull request into a Git repository by running: $ git pull https://github.com/dguy/kafka fix-compile-error Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1641.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1641 commit 60fa103e3a6fe1e4dd9788033e854886493016e0 Author: Damian Guy <damian.guy@gmail.com> Date: 2016-07-19T23:49:02Z fix compilation error
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/kafka/pull/1628

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1628
          Hide
          guozhang Guozhang Wang added a comment -

          Issue resolved by pull request 1628
          https://github.com/apache/kafka/pull/1628

          Show
          guozhang Guozhang Wang added a comment - Issue resolved by pull request 1628 https://github.com/apache/kafka/pull/1628
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user dguy opened a pull request:

          https://github.com/apache/kafka/pull/1628

          KAFKA-3812 State store locking is incorrect

          Move all state directory creation/locking/unlocking/cleaning to a single class. Don't release the channel until the lock is released. Refactor code to make use of new class

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/dguy/kafka kafka-3812

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/kafka/pull/1628.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1628


          commit b4547162d5d07cd471c1ccf9e84ed57c947bf2e4
          Author: Damian Guy <damian.guy@gmail.com>
          Date: 2016-07-17T16:29:20Z

          move locking/cleaning/creating/unlocking of task directories into a single class. refactor code to make use of it


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/1628 KAFKA-3812 State store locking is incorrect Move all state directory creation/locking/unlocking/cleaning to a single class. Don't release the channel until the lock is released. Refactor code to make use of new class You can merge this pull request into a Git repository by running: $ git pull https://github.com/dguy/kafka kafka-3812 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1628.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1628 commit b4547162d5d07cd471c1ccf9e84ed57c947bf2e4 Author: Damian Guy <damian.guy@gmail.com> Date: 2016-07-17T16:29:20Z move locking/cleaning/creating/unlocking of task directories into a single class. refactor code to make use of it

            People

            • Assignee:
              damianguy Damian Guy
              Reporter:
              enothereska Eno Thereska
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development