[HBASE-18312] Ineffective handling of FileNotFoundException in FileLink$FileLinkInputStream.tryOpen()

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0, 2.0.0-alpha-2, 2.0.0
• Hadoop Flags: Reviewed
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 04/Jul/17 03:08
• Updated: 01/Aug/18 06:20
• Resolved: 06/Jul/17 15:26

Description

Found the following in region server log:

2017-07-03 11:22:04,669 WARN  [regionserver/a.b.c.d:16020-shortCompactions-1499094046361] retry.RetryInvocationHandler: Exception while       invoking ClientNamenodeProtocolTranslatorPB.getBlockLocations over e.f.g.h:8020. Not retrying because try once and fail.
org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File does not exist: /hbase/data/default/X/4d61af9d1cbcc5fe2a5cbddbfc92fe7e/ K/47222a9cbd294f499f49de92ecf330ee
  at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
  at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
...
  at org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.tryOpen(FileLink.java:291)
  at org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.<init>(FileLink.java:122)
  at org.apache.hadoop.hbase.io.FileLink$FileLinkInputStream.<init>(FileLink.java:113)
  at org.apache.hadoop.hbase.io.FileLink.open(FileLink.java:404)
  at org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.<init>(FSDataInputStreamWrapper.java:98)
  at org.apache.hadoop.hbase.io.FSDataInputStreamWrapper.<init>(FSDataInputStreamWrapper.java:83)

Here is related code:

    private FSDataInputStream tryOpen() throws IOException {
      for (Path path: fileLink.getLocations()) {
...
        } catch (FileNotFoundException e) {
          // Try another file location
        }

The intention is to try possible locations for the linked file.
However, RemoteException was the exception encountered. This makes the above catch clause ineffective.