Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2002

Incorrect computation of needed blocks in getTurnOffTip()

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0, 0.23.0
    • Component/s: namenode
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      SafeModeInfo.getTurnOffTip() under-reports the number of blocks needed to reach the safemode threshold.

      1. hdfs-2002.patch
        2 kB
        Matthias Eckert
      2. testsafemode.patch
        1.0 kB
        Plamen Jeliazkov
      3. testsafemode.patch
        3 kB
        Plamen Jeliazkov
      4. HADOOP-2002_TRUNK.patch
        3 kB
        Plamen Jeliazkov

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #78 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/78/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #78 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/78/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #64 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/64/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #64 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/64/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #141 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/141/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #141 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/141/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #132 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/132/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #132 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/132/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #131 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/131/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #131 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/131/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196229 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #884 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/884/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #884 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/884/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #105 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/105/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195870
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/branches/branch-0.22/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #105 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/105/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195870 Files : /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt /hadoop/common/branches/branch-0.22/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.22/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #850 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/850/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #850 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/850/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1247 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1247/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1247 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1247/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Konstantin Shvachko added a comment -

          I just committed this. Thank you Plamen.

          Show
          Konstantin Shvachko added a comment - I just committed this. Thank you Plamen.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1222 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1222/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1222 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1222/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1297 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1297/)
          HDFS-2002. Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov.

          shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1297 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1297/ ) HDFS-2002 . Incorrect computation of needed blocks in getTurnOffTip(). Contributed by Plamen Jeliazkov. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1195869 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSafeMode.java
          Hide
          Konstantin Shvachko added a comment -

          +1 test failures are not related to the patch.

          Show
          Konstantin Shvachko added a comment - +1 test failures are not related to the patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12501572/HADOOP-2002_TRUNK.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hdfs.TestAbandonBlock
          org.apache.hadoop.cli.TestHDFSCLI

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1502//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1502//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501572/HADOOP-2002_TRUNK.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestAbandonBlock org.apache.hadoop.cli.TestHDFSCLI +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1502//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1502//console This message is automatically generated.
          Hide
          Plamen Jeliazkov added a comment -

          Sorry for the long wait. Konstantin had confirmed with me that my prior merged patch was now working and asked that I create a patch for trunk.

          This patch is a patch for trunk; exact same changes just mapped to trunks code.

          Show
          Plamen Jeliazkov added a comment - Sorry for the long wait. Konstantin had confirmed with me that my prior merged patch was now working and asked that I create a patch for trunk. This patch is a patch for trunk; exact same changes just mapped to trunks code.
          Hide
          Plamen Jeliazkov added a comment -

          Interesting – I don't see a TestSafeMode failure in there. I can't see how Timeout failures could be a result of a test expectation change... could be some sort of other latent bug or just a bad run of the test cases?

          Show
          Plamen Jeliazkov added a comment - Interesting – I don't see a TestSafeMode failure in there. I can't see how Timeout failures could be a result of a test expectation change... could be some sort of other latent bug or just a bad run of the test cases?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12486365/testsafemode.patch
          against trunk revision 1145843.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade
          org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp
          org.apache.hadoop.hdfs.server.datanode.TestDatanodeRestart
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
          org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool
          org.apache.hadoop.hdfs.server.datanode.TestMulitipleNNDataBlockScanner
          org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes
          org.apache.hadoop.hdfs.server.datanode.TestTransferRbw
          org.apache.hadoop.hdfs.server.datanode.TestWriteToReplica
          org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
          org.apache.hadoop.hdfs.server.namenode.TestAllowFormat
          org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks
          org.apache.hadoop.hdfs.server.namenode.TestBlockTokenWithDFS
          org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
          org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
          org.apache.hadoop.hdfs.server.namenode.TestClusterId
          org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp
          org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode
          org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus
          org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream
          org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
          org.apache.hadoop.hdfs.server.namenode.TestFsck
          org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader
          org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat
          org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
          org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
          org.apache.hadoop.hdfs.server.namenode.TestMetaSave
          org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs
          org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper
          org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean
          org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
          org.apache.hadoop.hdfs.server.namenode.TestNodeCount
          org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
          org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi
          org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
          org.apache.hadoop.hdfs.server.namenode.TestStreamFile
          org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage
          org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
          org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
          org.apache.hadoop.hdfs.tools.TestGetGroups
          org.apache.hadoop.security.TestRefreshUserMappings
          org.apache.hadoop.tools.TestJMXGet

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/917//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/917//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/917//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486365/testsafemode.patch against trunk revision 1145843. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade org.apache.hadoop.hdfs.server.datanode.TestDatanodeJsp org.apache.hadoop.hdfs.server.datanode.TestDatanodeRestart org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration org.apache.hadoop.hdfs.server.datanode.TestDeleteBlockPool org.apache.hadoop.hdfs.server.datanode.TestMulitipleNNDataBlockScanner org.apache.hadoop.hdfs.server.datanode.TestRefreshNamenodes org.apache.hadoop.hdfs.server.datanode.TestTransferRbw org.apache.hadoop.hdfs.server.datanode.TestWriteToReplica org.apache.hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps org.apache.hadoop.hdfs.server.namenode.TestAllowFormat org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks org.apache.hadoop.hdfs.server.namenode.TestBlockTokenWithDFS org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens org.apache.hadoop.hdfs.server.namenode.TestClusterId org.apache.hadoop.hdfs.server.namenode.TestCorruptFilesJsp org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode org.apache.hadoop.hdfs.server.namenode.TestDecommissioningStatus org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream org.apache.hadoop.hdfs.server.namenode.TestEditLogRace org.apache.hadoop.hdfs.server.namenode.TestFsck org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks org.apache.hadoop.hdfs.server.namenode.TestMetaSave org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs org.apache.hadoop.hdfs.server.namenode.TestNameNodeJspHelper org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker org.apache.hadoop.hdfs.server.namenode.TestNodeCount org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi org.apache.hadoop.hdfs.server.namenode.TestSecurityTokenEditLog org.apache.hadoop.hdfs.server.namenode.TestStreamFile org.apache.hadoop.hdfs.server.namenode.TestTransferFsImage org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer org.apache.hadoop.hdfs.tools.TestGetGroups org.apache.hadoop.security.TestRefreshUserMappings org.apache.hadoop.tools.TestJMXGet +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/917//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/917//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/917//console This message is automatically generated.
          Hide
          Plamen Jeliazkov added a comment -

          Oops, I think I needed to merge the hdfs-2002 patch and the testsafemode patch.

          Posted patch is both merged.

          Show
          Plamen Jeliazkov added a comment - Oops, I think I needed to merge the hdfs-2002 patch and the testsafemode patch. Posted patch is both merged.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12486349/testsafemode.patch
          against trunk revision 1145843.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.hdfs.server.namenode.TestSafeMode

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/916//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/916//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/916//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486349/testsafemode.patch against trunk revision 1145843. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.namenode.TestSafeMode +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/916//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/916//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/916//console This message is automatically generated.
          Hide
          Plamen Jeliazkov added a comment -

          Patch for review – fixing the TestSafeMode JUnit testcase to accept the new value.

          Show
          Plamen Jeliazkov added a comment - Patch for review – fixing the TestSafeMode JUnit testcase to accept the new value.
          Hide
          Jakob Homan added a comment -

          Matthias - looks like TestSafeMode is checking the output of the getTip call. Can you update the test to expect the new values? Thanks. -jg

          Show
          Jakob Homan added a comment - Matthias - looks like TestSafeMode is checking the output of the getTip call. Can you update the test to expect the new values? Thanks. -jg
          Hide
          Konstantin Shvachko added a comment -

          Looks like TestSafeMode failure is related to the patch.

          Show
          Konstantin Shvachko added a comment - Looks like TestSafeMode failure is related to the patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12481935/hdfs-2002.patch
          against trunk revision 1133476.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.cli.TestHDFSCLI
          org.apache.hadoop.hdfs.server.namenode.TestBackupNode
          org.apache.hadoop.hdfs.server.namenode.TestSafeMode

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/752//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/752//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/752//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481935/hdfs-2002.patch against trunk revision 1133476. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.cli.TestHDFSCLI org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.server.namenode.TestSafeMode +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/752//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/752//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/752//console This message is automatically generated.
          Hide
          Matthias Eckert added a comment -

          Log a warning if the threshold is larger than 1.
          Correct the number of remaining nodes and blocks.

          Show
          Matthias Eckert added a comment - Log a warning if the threshold is larger than 1. Correct the number of remaining nodes and blocks.
          Hide
          Konstantin Shvachko added a comment -

          In case there are 3 blocks in the system with the default threshold 0.9990, and with no replicas reported yet the safemode tip says:
          Safe mode is ON. The reported blocks 0 needs additional 2 blocks to reach the threshold 0.9990 of total blocks 3. Safe mode will be turned off automatically.
          While it should report one more needs additional 3 blocks.

          And when the threshold is set larger than one, say 1.5, the numbers don't make any sense at all.

          Show
          Konstantin Shvachko added a comment - In case there are 3 blocks in the system with the default threshold 0.9990, and with no replicas reported yet the safemode tip says: Safe mode is ON. The reported blocks 0 needs additional 2 blocks to reach the threshold 0.9990 of total blocks 3. Safe mode will be turned off automatically. While it should report one more needs additional 3 blocks . And when the threshold is set larger than one, say 1.5, the numbers don't make any sense at all.

            People

            • Assignee:
              Plamen Jeliazkov
              Reporter:
              Konstantin Shvachko
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development