[HBASE-18377] Error handling for FileNotFoundException should consider RemoteException in openReader()

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0, 2.0.0-alpha-2, 2.0.0
• Hadoop Flags: Reviewed
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 13/Jul/17 15:31
• Updated: 01/Aug/18 06:21
• Resolved: 18/Jul/17 13:56

Description

In region server log, I observed the following:

org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File does not exist: /apps/hbase/data/WALs/lx.p.com,16020,1497300923131/497300923131. default.1497302873178
  at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
  at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
  at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1860)
  at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1831)
  at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1744)
...
  at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
  at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:326)
  at org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:162)
  at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:782)
  at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:293)
  at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:267)
  at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:255)
  at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:414)
  at org.apache.hadoop.hbase.replication.regionserver.ReplicationWALReaderManager.openReader(ReplicationWALReaderManager.java:69)
  at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:605)
  at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:364)

We have code in ReplicationSource#openReader() which is supposed to handle FileNotFoundException but RemoteException wrapping FileNotFoundException was missed.