[HBASE-16] [hbase] IOException: File does not exist

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Component/s: regionserver
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 13/Dec/07 04:56
• Updated: 02/May/13 02:29
• Resolved: 24/Mar/08 15:58

Description

Two fellas today on two unrelated clusters had versions of the below:

      <bryanduxbury>	2007-12-12 08:28:22,235 ERROR org.apache.hadoop.hbase.HRegionServer: Compaction failed for region spider_pages,10_149317711,1197468834206
[13:01]	<bryanduxbury>	java.io.IOException: java.io.IOException: File does not exist
[13:01]	<bryanduxbury>	at org.apache.hadoop.dfs.FSDirectory.getFileInfo(FSDirectory.java:489)
[13:01]	<bryanduxbury>	at org.apache.hadoop.dfs.FSNamesystem.getFileInfo(FSNamesystem.java:1360)
[13:01]	<bryanduxbury>	at org.apache.hadoop.dfs.NameNode.getFileInfo(NameNode.java:428)
[13:01]	<bryanduxbury>	at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
[13:01]	<bryanduxbury>	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[13:01]	<bryanduxbury>	at java.lang.reflect.Method.invoke(Method.java:597)
[13:01]	<bryanduxbury>	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:379)
[13:01]	<bryanduxbury>	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:596)
[13:01]	<bryanduxbury>	at sun.reflect.GeneratedConstructorAccessor10.newInstance(Unknown Source)
[13:01]	<bryanduxbury>	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
[13:01]	<bryanduxbury>	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
[13:01]	<bryanduxbury>	at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
[13:01]	<bryanduxbury>	at org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteExceptionHandler.java:48)
[13:01]	<bryanduxbury>	at org.apache.hadoop.hbase.HRegionServer$Compactor.run(HRegionServer.java:385)

Odd is that the file thats missing's name is not cited.

The other instance showed in the webui. Seemed to be problem with an HStoreFile in.META. region. I was unable to select content from the .META. table – it was returning null rows.

In both cases a restart fixed things again.

Since all state is out in hdfs and the in-memory maps are made from the hdfs state, something must not be getting updated on compaction/split or flush.

Issue Links

• depends upon: HADOOP-2432 If HDFS is going to throw an exception "File does not exist" it should include the name of the file