Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.3.0
-
None
-
None
Description
The new map output http client uses java.net.URLConnection to fetch the data file. However under Java 1.4 there is no way to specify a timeout and it is set to infinite (or if not infinite at least 12 hours). This causes reduce tasks to get "stuck" in the "reduce > copy" phase even after the "Task failed to report status for 600 seconds. Killing." message.
I will add the code in the ReduceTaskRunner to make sure that copies in-flight don't get stuck, but this is another point where a switch to java 1.5 would be helpful. Under 1.5 I could set the timeout on the connection and the read would timeout after the given interval and my entire change would be local to MapOutputLocation.copyFile.
For now, I'll assume that we need to maintain support for 1.4 and make the corresponding fix, but I'm grumbling...