As per the mailing thread with the heading
'Handling read failures during recovery' we found this problem.
As part of split Logs the HMaster calls Namenode recovery. The recovery is an asynchronous process.
Even though client is getting the updated block info from Namenode on first
read failure, client is discarding the new info and using the old info only
to retrieve the data from datanode. So, all the read
retries are failing. [Method parameter reassignment - Not reflected in
In HMaster code we tend to wait for 1sec. But if the recovery had some failure then split log may not happen and may lead to dataloss.
So may be we need to decide upon the actual delay that needs to be introduced once Hmaster calls NN recovery.
|Field||Original Value||New Value|
|Priority||Major [ 3 ]||Critical [ 2 ]|
|Status||Open [ 1 ]||Resolved [ 5 ]|
|Fix Version/s||0.95.1 [ 12324288 ]|
|Resolution||Fixed [ 1 ]|
|Status||Resolved [ 5 ]||Closed [ 6 ]|
|Transition||Time In Source Status||Execution Times||Last Executer||Last Execution Date|
|686d 17h 3m||1||Nicolas Liochon||25/Jun/13 10:52|
|90d 8h 15m||1||stack||23/Sep/13 19:08|