Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Won't Fix
-
0.9.2
-
None
-
None
-
None
Description
randomwriter with speculative execution turned 'on' fails with too many errors:
2006-12-25 06:50:38,880 INFO org.apache.hadoop.mapred.TaskInProgress: Error from task_0002_m_002175_1: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /randomwriter/input/part000522 for DFSClient_task_0002_m_002101_1 on client XXX because pendingCreates is non-null.
2006-12-25 06:50:48,203 INFO org.apache.hadoop.mapred.TaskInProgress: Error from task_0002_m_002213_1: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /randomwriter/input/part000623 for DFSClient_task_0002_m_002213_1 on client XXX because pendingCreates is non-null.
2006-12-25 06:50:49,917 INFO org.apache.hadoop.mapred.TaskInProgress: Error from task_0002_m_002211_1: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /randomwriter/input/part000621 for DFSClient_task_0002_m_002211_1 on client XXX because pendingCreates is non-null.
2006-12-25 06:50:52,812 INFO org.apache.hadoop.mapred.TaskInProgress: Error from task_0002_m_002247_1: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /randomwriter/input/part000654 for DFSClient_task_0002_m_002247_1 on client XXX because pendingCreates is non-null.
I suspect this is due to the the fact that randomwriter doesn't use the PhasedFileSystem.
-
Unrelated note: can we add the 'examples' Component to this project here on jira? I've put in 'mapred' for this issue for now, please feel free to correct me. Thanks!