Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8850

VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The VolumeScanner threads inside the BlockScanner exit with an exception if there is no block pool to be scanned but there are suspicious blocks.

      1. HDFS-8850.001.patch
        1 kB
        Colin P. McCabe

        Issue Links

          Activity

          Hide
          hitliuyi Yi Liu added a comment -

          Yes, you are right. +1 pending Jenkins.

          Show
          hitliuyi Yi Liu added a comment - Yes, you are right. +1 pending Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 11s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 40s There were no new javac warning messages.
          +1 javadoc 9m 40s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 20s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 23s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 2s Pre-build of native portion
          -1 hdfs tests 158m 28s Tests failed in hadoop-hdfs.
              202m 14s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748576/HDFS-8850.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c3364ca
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11892/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11892/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11892/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 40s There were no new javac warning messages. +1 javadoc 9m 40s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 20s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 2s Pre-build of native portion -1 hdfs tests 158m 28s Tests failed in hadoop-hdfs.     202m 14s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748576/HDFS-8850.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c3364ca hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11892/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11892/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11892/console This message was automatically generated.
          Hide
          hitliuyi Yi Liu added a comment -

          Test failure is unrelated. Committed to trunk and branch-2.
          Thanks Colin P. McCabe for the contribution.

          Show
          hitliuyi Yi Liu added a comment - Test failure is unrelated. Committed to trunk and branch-2. Thanks Colin P. McCabe for the contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8259 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8259/)
          HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8259 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8259/ ) HDFS-8850 . VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #277 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/277/)
          HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #277 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/277/ ) HDFS-8850 . VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1007 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1007/)
          HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1007 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1007/ ) HDFS-8850 . VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2204 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2204/)
          HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2204 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2204/ ) HDFS-8850 . VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2223 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2223/)
          HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2223 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2223/ ) HDFS-8850 . VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #266 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/266/)
          HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #266 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/266/ ) HDFS-8850 . VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #274 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/274/)
          HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #274 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/274/ ) HDFS-8850 . VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) (yliu: rev 9a08999906824e8982ec3b47315f9b960474186a) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/VolumeScanner.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          sjlee0 Sangjin Lee added a comment -

          Removing the 2.6.1-candidate label, as this is an issue for a class that exists only in 2.7 and as such I don't think it applies to 2.6. Let me know if you think this issue exists in 2.6.

          Show
          sjlee0 Sangjin Lee added a comment - Removing the 2.6.1-candidate label, as this is an issue for a class that exists only in 2.7 and as such I don't think it applies to 2.6. Let me know if you think this issue exists in 2.6.
          Hide
          xuchenCN Xu Chen added a comment -

          +1 LGTM

          Show
          xuchenCN Xu Chen added a comment - +1 LGTM
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Adding back 2.6.1-candidate label for tracking. Will remove it once 2.6.1 is done.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Adding back 2.6.1-candidate label for tracking. Will remove it once 2.6.1 is done.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          About to make an RC for 2.6.1. Dropping the 2.6.1 label as we have decided to drop this. Adding it to 2.7.2 though.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - About to make an RC for 2.6.1. Dropping the 2.6.1 label as we have decided to drop this. Adding it to 2.7.2 though.
          Hide
          shahrs87 Rushabh S Shah added a comment -

          Yi Liu, Colin P. McCabe: Does it make sense to commit to 2.7.2 ?

          Show
          shahrs87 Rushabh S Shah added a comment - Yi Liu , Colin P. McCabe : Does it make sense to commit to 2.7.2 ?
          Hide
          kihwal Kihwal Lee added a comment -

          Cherry-picked to branch-2.7.

          Show
          kihwal Kihwal Lee added a comment - Cherry-picked to branch-2.7.
          Hide
          cmccabe Colin P. McCabe added a comment -

          I think this makes sense for 2.7.2. Thanks, all.

          Show
          cmccabe Colin P. McCabe added a comment - I think this makes sense for 2.7.2. Thanks, all.

            People

            • Assignee:
              cmccabe Colin P. McCabe
              Reporter:
              cmccabe Colin P. McCabe
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development