[HDFS-13511] Provide specialized exception when block length cannot be obtained

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.1
• Hadoop Flags: Reviewed
• Assignee: Gabor Bota
• Reporter: Ted Yu
• Created: 27/Apr/18 15:38
• Updated: 12/Jun/18 18:18
• Resolved: 06/Jun/18 04:25

Description

In downstream project, I saw the following code:

        FSDataInputStream inputStream = hdfs.open(new Path(path));
...
        if (options.getRecoverFailedOpen() && dfs != null && e.getMessage().toLowerCase()
            .startsWith("cannot obtain block length for")) {

The above tightly depends on the following in DFSInputStream#readBlockLength

    throw new IOException("Cannot obtain block length for " + locatedblock);

The check based on string matching is brittle in production deployment.

After discussing with Steve Loughran, better approach is to introduce specialized IOException, e.g. CannotObtainBlockLengthException so that downstream project doesn't have to rely on string matching.

Issue Links

• is related to: HDFS-11711 DN should not delete the block On "Too many open files" Exception