Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.24.0, 0.23.3
    • Fix Version/s: 2.0.0-alpha
    • Component/s: namenode
    • Labels:
      None

      Description

      "Inconsistent synchronization of org.apache.hadoop.hdfs.server.namenode.NameNode.state; locked 50% of time". I think this was probably one of my recent cross-project HA patches in which the QA bot was unable to run. Will investigate.

      1. hdfs-3132.txt
        0.9 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1029 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1029/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1029 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1029/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #235 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/235/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682)

          Result = FAILURE
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #235 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/235/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #207 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/207/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682)

          Result = UNSTABLE
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #207 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/207/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682) Result = UNSTABLE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #994 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/994/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681)

          Result = FAILURE
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #994 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/994/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #731 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/731/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682)

          Result = ABORTED
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #731 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/731/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682) Result = ABORTED todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1933 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1933/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681)

          Result = ABORTED
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1933 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1933/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681) Result = ABORTED todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #722 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/722/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #722 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/722/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1997 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1997/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1997 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1997/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #712 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/712/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #712 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/712/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304682) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304682 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1923 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1923/)
          HDFS-3132. Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1923 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1923/ ) HDFS-3132 . Fix findbugs warning on HDFS trunk. Contributed by Todd Lipcon. (Revision 1304681) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1304681 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          Hide
          Todd Lipcon added a comment -

          Committed to trunk and 23

          Show
          Todd Lipcon added a comment - Committed to trunk and 23
          Hide
          Tsz Wo Nicholas Sze added a comment -

          BTW, +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - BTW, +1 patch looks good.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          There's not currently any easy way to run all the checks. And the problem with splitting into sub-projects is that, often, a patch in Common would actually break the build if it were committed before the associated patch in HDFS. I'd rather chance introducing a few warnings than break the build for everyone.

          Agree. If splitting would break the build, it is better to commit together.

          Buy why it is not easy to run the checks? I think it is easy at least to run findbugs.

          Show
          Tsz Wo Nicholas Sze added a comment - There's not currently any easy way to run all the checks. And the problem with splitting into sub-projects is that, often, a patch in Common would actually break the build if it were committed before the associated patch in HDFS. I'd rather chance introducing a few warnings than break the build for everyone. Agree. If splitting would break the build, it is better to commit together. Buy why it is not easy to run the checks? I think it is easy at least to run findbugs.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12519684/hdfs-3132.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.TestGetBlocks
          org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2083//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2083//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12519684/hdfs-3132.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.TestGetBlocks org.apache.hadoop.hdfs.server.namenode.TestValidateConfigurationSettings +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2083//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2083//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Patch makes NameNode.state volatile to quiet the warning. This should be sufficient since the actual complicated transitions are themselves synchronized.

          Show
          Todd Lipcon added a comment - Patch makes NameNode.state volatile to quiet the warning. This should be sufficient since the actual complicated transitions are themselves synchronized.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Is this broken by HDFS-3071?

          Show
          Tsz Wo Nicholas Sze added a comment - Is this broken by HDFS-3071 ?
          Hide
          Todd Lipcon added a comment -

          Please avoid cross-project patches and split the patches into sub-projects. Or, you have to run all the checks and tests before committing

          There's not currently any easy way to run all the checks. And the problem with splitting into sub-projects is that, often, a patch in Common would actually break the build if it were committed before the associated patch in HDFS. I'd rather chance introducing a few warnings than break the build for everyone.

          Regarding these particular patches, I originally put all of the code for HA in HDFS, but others argued we should try to build a generic framework (despite the fact that HDFS is the only implementer of all of these new Common pieces). So now we're left in this situation where any non-trivial changes to failover have to go cross-project.

          I guess you don't want to be known for adding findbugs or other warnings to Hadoop.

          Somehow I don't think that's what I'm known for... out of the 20+ things I've committed in the last month, a mistake rate of 5% doesn't seem so bad.

          Hopefully someone can spend some time working on cross-project test-patch which would ameliorate this issue.

          Show
          Todd Lipcon added a comment - Please avoid cross-project patches and split the patches into sub-projects. Or, you have to run all the checks and tests before committing There's not currently any easy way to run all the checks. And the problem with splitting into sub-projects is that, often, a patch in Common would actually break the build if it were committed before the associated patch in HDFS. I'd rather chance introducing a few warnings than break the build for everyone. Regarding these particular patches, I originally put all of the code for HA in HDFS, but others argued we should try to build a generic framework (despite the fact that HDFS is the only implementer of all of these new Common pieces). So now we're left in this situation where any non-trivial changes to failover have to go cross-project. I guess you don't want to be known for adding findbugs or other warnings to Hadoop. Somehow I don't think that's what I'm known for... out of the 20+ things I've committed in the last month, a mistake rate of 5% doesn't seem so bad. Hopefully someone can spend some time working on cross-project test-patch which would ameliorate this issue.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Please avoid cross-project patches and split the patches into sub-projects. Or, you have to run all the checks and tests before committing. I guess you don't want to be known for adding findbugs or other warnings to Hadoop.

          Show
          Tsz Wo Nicholas Sze added a comment - Please avoid cross-project patches and split the patches into sub-projects. Or, you have to run all the checks and tests before committing. I guess you don't want to be known for adding findbugs or other warnings to Hadoop.

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development