Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8845

DiskChecker should not traverse the entire tree

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      DiskChecker should not traverse entire tree because it's causing heavy disk load on checkDiskError()

        Issue Links

          Activity

          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Hi, Chang Li, after HDFS-6482, finalizedDir has two-level of subdirs (finalized/subdir0/subdir23/blk_1234). Would this change lose the coverage of checkDir() on these subdirs?

          Show
          eddyxu Lei (Eddy) Xu added a comment - Hi, Chang Li , after HDFS-6482 , finalizedDir has two-level of subdirs ( finalized/subdir0/subdir23/blk_1234 ). Would this change lose the coverage of checkDir() on these subdirs?
          Hide
          lichangleo Chang Li added a comment -

          Hi Lei (Eddy) Xu, thanks for comments. It's intentionally done for performance by a little bit trade off.

          Show
          lichangleo Chang Li added a comment - Hi Lei (Eddy) Xu , thanks for comments. It's intentionally done for performance by a little bit trade off.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 29s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 39s There were no new javac warning messages.
          +1 javadoc 9m 46s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 21s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 2s Pre-build of native portion
          -1 hdfs tests 159m 42s Tests failed in hadoop-hdfs.
              203m 51s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748230/HDFS-8845.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / d0e0ba8
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11880/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11880/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11880/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 29s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 39s There were no new javac warning messages. +1 javadoc 9m 46s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 21s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 2s Pre-build of native portion -1 hdfs tests 159m 42s Tests failed in hadoop-hdfs.     203m 51s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748230/HDFS-8845.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d0e0ba8 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11880/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11880/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11880/console This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          It'd be a little better for coverage if we tried looking at the file path that had the error, since it's possible there's an error on some subdir. Agree though that traversing the entire tree doesn't have much point though.

          Show
          andrew.wang Andrew Wang added a comment - It'd be a little better for coverage if we tried looking at the file path that had the error, since it's possible there's an error on some subdir. Agree though that traversing the entire tree doesn't have much point though.
          Hide
          cmccabe Colin P. McCabe added a comment - - edited

          That's an interesting point, Andrew Wang. However, the BlockScanner should be scheduled to rescan the block after the error occurs. So it should be reported to the NN relatively rapidly even without adding synchronous code that looks at the block file path.

          +1 for the latest patch. Will commit in a day or two if there are no more comments

          Show
          cmccabe Colin P. McCabe added a comment - - edited That's an interesting point, Andrew Wang . However, the BlockScanner should be scheduled to rescan the block after the error occurs. So it should be reported to the NN relatively rapidly even without adding synchronous code that looks at the block file path. +1 for the latest patch. Will commit in a day or two if there are no more comments
          Hide
          cmccabe Colin P. McCabe added a comment -

          Committed to 2.8. Thanks, Chang Li.

          Show
          cmccabe Colin P. McCabe added a comment - Committed to 2.8. Thanks, Chang Li .
          Hide
          andrew.wang Andrew Wang added a comment -

          Yea sounds good to me too, Colin explained to me offline that the BlockScanner processes "suspect" blocks like these first, so anything that would get caught by a path-checker will get quickly caught by the BlockScanner. Thanks all.

          Show
          andrew.wang Andrew Wang added a comment - Yea sounds good to me too, Colin explained to me offline that the BlockScanner processes "suspect" blocks like these first, so anything that would get caught by a path-checker will get quickly caught by the BlockScanner. Thanks all.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8315/)
          HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8315/ ) HDFS-8845 . DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1021 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1021/)
          HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1021 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1021/ ) HDFS-8845 . DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/291/)
          HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/291/ ) HDFS-8845 . DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #288 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/288/)
          HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #288 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/288/ ) HDFS-8845 . DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2237 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2237/)
          HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2237 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2237/ ) HDFS-8845 . DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2218 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2218/)
          HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2218 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2218/ ) HDFS-8845 . DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/280/)
          HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/280/ ) HDFS-8845 . DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) (cmccabe: rev ec183faadcf7edaf432aca3b25d24215d505c2ec) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
          Hide
          vinayrpet Vinayakumar B added a comment -

          Do any body thinks this should be merged to 2.7 and 2.6?
          cc. Junping Du, Kihwal Lee ?

          I am +1 for merging.

          Show
          vinayrpet Vinayakumar B added a comment - Do any body thinks this should be merged to 2.7 and 2.6? cc. Junping Du , Kihwal Lee ? I am +1 for merging.
          Hide
          kihwal Kihwal Lee added a comment -

          +1 for merging to 2.7.

          Show
          kihwal Kihwal Lee added a comment - +1 for merging to 2.7.
          Hide
          kihwal Kihwal Lee added a comment -

          Cherry-picked to 2.7.

          Show
          kihwal Kihwal Lee added a comment - Cherry-picked to 2.7.
          Hide
          ctrezzo Chris Trezzo added a comment -

          Adding 2.6.5 to the target versions with the intention of backporting this to branch-2.6. Please let me know if you think otherwise. Thanks!

          Show
          ctrezzo Chris Trezzo added a comment - Adding 2.6.5 to the target versions with the intention of backporting this to branch-2.6. Please let me know if you think otherwise. Thanks!
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.
          Hide
          sjlee0 Sangjin Lee added a comment -

          Cherry-picked it into 2.6.5 (trivial).

          Show
          sjlee0 Sangjin Lee added a comment - Cherry-picked it into 2.6.5 (trivial).
          Hide
          ctrezzo Chris Trezzo added a comment -

          Thanks!

          Show
          ctrezzo Chris Trezzo added a comment - Thanks!

            People

            • Assignee:
              lichangleo Chang Li
              Reporter:
              lichangleo Chang Li
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development