Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1063

MiniDFSCluster exists a race condition that lead to data node resources are not properly released

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.12.0
    • 0.13.0
    • test
    • None
    • Resolved race condition in shutting down MiniDFSCluster data node that prevented resources from being deallocated properly.

    Description

      In MiniDFSCluster, there is a possibility that a data node gets shutted down before it is constructed. This leads to the situation that the data node's resources are not properly released. In Cygwin I observe that the data node directory is still kept locked after a miniDFSCluster is shut down.

      Attachments

        1. miniDFSCluster.patch
          3 kB
          Hairong Kuang

        Issue Links

          Activity

            hairong Hairong Kuang added a comment -

            This patch removes the race condition in MiniDFSCluster. It guarantees that resources are released after a mini dfs cluster shuts down.

            hairong Hairong Kuang added a comment - This patch removes the race condition in MiniDFSCluster. It guarantees that resources are released after a mini dfs cluster shuts down.
            cutting Doug Cutting added a comment -

            I just committed this. Thanks, Hairong.

            cutting Doug Cutting added a comment - I just committed this. Thanks, Hairong.

            I'm getting a NullPointerException in MiniDFSCluster.DataNodeRunner.run()
            node.shutdown();
            because node is null. Should it be just
            shutdown();

            shv Konstantin Shvachko added a comment - I'm getting a NullPointerException in MiniDFSCluster.DataNodeRunner.run() node.shutdown(); because node is null. Should it be just shutdown();

            People

              hairong Hairong Kuang
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: