[MAHOUT-1325] MapReduce unit tests cannot be run in parallel

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.8
• Fix Version/s: 0.9
• Component/s: classic

Environment

ubuntu1 Apache Jenkins CI node

• Assignee: Suneel Marthi
• Reporter: Stevo Slavić
• Created: 04/Sep/13 18:25
• Updated: 31/Jan/24 22:17
• Resolved: 13/Nov/13 21:43

Description

Mahout-Quality build job execution #2225 failed because of this test.
Relevant build log:

[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ mahout-integration ---
[INFO] Surefire report directory: /home/jenkins/jenkins-slave/workspace/Mahout-Quality/trunk/integration/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.mahout.utils.vectors.lucene.LuceneIterableTest
Running org.apache.mahout.utils.email.MailProcessorTest
Running org.apache.mahout.clustering.cdbw.TestCDbwEvaluator
Running org.apache.mahout.utils.regex.RegexMapperTest
Running org.apache.mahout.utils.vectors.io.VectorWriterTest
Running org.apache.mahout.utils.nlp.collocations.llr.BloomTokenFilterTest
Running org.apache.mahout.utils.vectors.lucene.CachedTermInfoTest
Running org.apache.mahout.clustering.TestClusterEvaluator
Running org.apache.mahout.utils.regex.RegexUtilsTest
Running org.apache.mahout.utils.vectors.lucene.DriverTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.17 sec - in org.apache.mahout.utils.regex.RegexUtilsTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.412 sec - in org.apache.mahout.utils.email.MailProcessorTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.344 sec - in org.apache.mahout.utils.nlp.collocations.llr.BloomTokenFilterTest
Running org.apache.mahout.utils.vectors.csv.CSVVectorIteratorTest
Running org.apache.mahout.clustering.TestClusterDumper
Running org.apache.mahout.utils.vectors.VectorHelperTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.601 sec - in org.apache.mahout.utils.vectors.csv.CSVVectorIteratorTest
Running org.apache.mahout.utils.vectors.arff.ARFFTypeTest
Running org.apache.mahout.utils.vectors.arff.ARFFVectorIterableTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.383 sec - in org.apache.mahout.utils.vectors.VectorHelperTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.332 sec - in org.apache.mahout.utils.vectors.lucene.LuceneIterableTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.252 sec - in org.apache.mahout.utils.vectors.arff.ARFFTypeTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.369 sec - in org.apache.mahout.utils.vectors.lucene.CachedTermInfoTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.415 sec - in org.apache.mahout.utils.vectors.arff.ARFFVectorIterableTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.5 sec - in org.apache.mahout.utils.regex.RegexMapperTest
Running org.apache.mahout.utils.vectors.arff.DriverTest
Running org.apache.mahout.utils.Bump125Test
Running org.apache.mahout.utils.SplitInputTest
Running org.apache.mahout.utils.TestConcatenateVectorsJob
Running org.apache.mahout.utils.vectors.arff.MapBackedARFFModelTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.043 sec - in org.apache.mahout.utils.vectors.io.VectorWriterTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.362 sec - in org.apache.mahout.utils.Bump125Test
Running org.apache.mahout.cf.taste.impl.similarity.jdbc.MySQLJDBCInMemoryItemSimilarityTest
Running org.apache.mahout.text.SequenceFilesFromLuceneStorageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.304 sec - in org.apache.mahout.utils.vectors.arff.MapBackedARFFModelTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.887 sec - in org.apache.mahout.utils.vectors.arff.DriverTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.486 sec - in org.apache.mahout.cf.taste.impl.similarity.jdbc.MySQLJDBCInMemoryItemSimilarityTest
Running org.apache.mahout.text.TestSequenceFilesFromDirectory
Running org.apache.mahout.text.LuceneStorageConfigurationTest
Running org.apache.mahout.text.LuceneSegmentInputFormatTest
Running org.apache.mahout.text.SequenceFilesFromMailArchivesTest
Running org.apache.mahout.text.MailArchivesClusteringAnalyzerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.265 sec - in org.apache.mahout.text.MailArchivesClusteringAnalyzerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.8 sec - in org.apache.mahout.utils.TestConcatenateVectorsJob
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.097 sec - in org.apache.mahout.text.LuceneStorageConfigurationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.902 sec - in org.apache.mahout.utils.vectors.lucene.DriverTest
Running org.apache.mahout.text.LuceneSegmentRecordReaderTest
Running org.apache.mahout.text.LuceneSegmentInputSplitTest
Running org.apache.mahout.text.SequenceFilesFromLuceneStorageMRJobTest
Running org.apache.mahout.text.SequenceFilesFromLuceneStorageDriverTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.237 sec - in org.apache.mahout.text.LuceneSegmentInputFormatTest
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.122 sec <<< FAILURE! - in org.apache.mahout.text.SequenceFilesFromMailArchivesTest
testMapReduce(org.apache.mahout.text.SequenceFilesFromMailArchivesTest)  Time elapsed: 2.333 sec  <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:743)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:555)
	at org.junit.Assert.assertEquals(Assert.java:542)
	at org.apache.mahout.text.SequenceFilesFromMailArchivesTest.testMapReduce(SequenceFilesFromMailArchivesTest.java:153)

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.74 sec - in org.apache.mahout.text.TestSequenceFilesFromDirectory
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.172 sec - in org.apache.mahout.text.LuceneSegmentInputSplitTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.85 sec - in org.apache.mahout.text.LuceneSegmentRecordReaderTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.571 sec - in org.apache.mahout.clustering.TestClusterDumper
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.021 sec - in org.apache.mahout.text.SequenceFilesFromLuceneStorageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.121 sec - in org.apache.mahout.text.SequenceFilesFromLuceneStorageMRJobTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.613 sec - in org.apache.mahout.text.SequenceFilesFromLuceneStorageDriverTest
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.246 sec - in org.apache.mahout.utils.SplitInputTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.07 sec - in org.apache.mahout.clustering.cdbw.TestCDbwEvaluator
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.224 sec - in org.apache.mahout.clustering.TestClusterEvaluator

Results :

Failed tests: 
  SequenceFilesFromMailArchivesTest.testMapReduce:153->Assert.assertEquals:542->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88 expected:<1> but was:<0>

Tests run: 95, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Mahout Build Tools ................................ SUCCESS [8.296s]
[INFO] Apache Mahout ..................................... SUCCESS [2.731s]
[INFO] Mahout Math ....................................... SUCCESS [1:38.594s]
[INFO] Mahout Core ....................................... SUCCESS [4:44.809s]
[INFO] Mahout Integration ................................ FAILURE [26.458s]
[INFO] Mahout Examples ................................... SKIPPED
[INFO] Mahout Release Package ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7:03.463s
[INFO] Finished at: Tue Sep 03 22:40:41 UTC 2013
[INFO] Final Memory: 62M/447M
[INFO] ------------------------------------------------------------------------

Issue Links

• depends upon: MAPREDUCE-5367 Local jobs all use same local working directory
• is duplicated by: MAHOUT-1321 TestSequenceFilesFromDirectory.testSequenceFileFromDirectoryMapReduce is unstable
• is duplicated by: MAHOUT-1322 TestDistributedRowMatrix.testTranspose is unstable
• is duplicated by: MAHOUT-1323 ParallelALSFactorizationJobTest.completeJobToyExample is unstable
• is duplicated by: MAHOUT-1324 SequenceFilesFromLuceneStorageMRJobTest.testRun is unstable
• is related to: MAHOUT-1200 Mahout tests depend on writing to /tmp/hadoop-$user