[HADOOP-1556] 9 unit test failures: file.out.index already exists

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Assignee: Devaraj Das
• Reporter: Nigel Daley
• Created: 02/Jul/07 04:06
• Updated: 08/Jul/09 16:52
• Resolved: 10/Jul/07 19:54

Description

In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:

java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)

I'm concerned this could be a race conditrion. The unit tests are:
org.apache.hadoop.fs.TestDFSIO.testIOs
org.apache.hadoop.fs.TestFileSystem.testFs
org.apache.hadoop.mapred.TestAggregates.testAggregates
org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
org.apache.hadoop.mapred.TestLocalMRNotification.testMR
org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
org.apache.hadoop.mapred.TestMapRed.testMapred
org.apache.hadoop.mapred.TestMapRed.testCompression
org.apache.hadoop.record.TestRecordMR.testMapred