[HDFS-7764] DirectoryScanner shouldn't abort the scan if one directory had an error

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.9.0, 3.0.0-alpha1, 2.8.2
• Component/s: datanode
• Assignee: Rakesh Radhakrishnan
• Reporter: Rakesh Radhakrishnan
• Created: 10/Feb/15 07:51
• Updated: 25/Aug/17 18:24
• Resolved: 29/Jan/16 04:12

Description

If there is an exception while preparing the ScanInfo for the blocks in the directory, DirectoryScanner is immediately throwing exception and coming out of the current scan cycle. The idea of this jira is to discuss & improve the exception handling mechanism.

DirectoryScanner.java

    for (Entry<Integer, Future<ScanInfoPerBlockPool>> report :
        compilersInProgress.entrySet()) {
      try {
        dirReports[report.getKey()] = report.getValue().get();
      } catch (Exception ex) {
        LOG.error("Error compiling report", ex);
        // Propagate ex to DataBlockScanner to deal with
        throw new RuntimeException(ex);
      }