[#HADOOP-5920] TestJobHistory fails some times.

Hadoop Common

TestJobHistory fails some times.

Created: 27/May/09 09:47 AM Updated: 08/Jul/09 04:53 PM

Return to search

Component/s:

None

Affects Version/s:

0.21.0

Fix Version/s:

0.20.1

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
HADOOP-5920-v1.1-branch0.20.patch	2009-06-18 03:35 AM	Amar Kamat	2 kB
HADOOP-5920-v1.1.patch	2009-06-12 08:28 AM	Amar Kamat	2 kB

Hadoop Flags:	Reviewed
Release Note:	TestJobHistory fails as jobtracker is restarted very fast (within a minute) and history files from earlier testcases were not cleaned up. This patch cleans up the history-dir and mapred-system-dir after every test.
Resolution Date:	17/Jun/09 12:29 PM
Tags:	ygridqa

Description

« Hide

TestJobHistory fails sometimes with following Error:
User log file file:/home/hudson/hudson-slave/workspace/Hadoop-Patch-vesta.apache.org/trunk/build/test/data/succeed/output1/_logs/history/localhost_1243294208119_job_200905252330_0001_hudson_test-job-succeed does not exist

Correspoding failure on the hudson is @ http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/398/testReport/

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

Todd Lipcon added a comment - 27/May/09 08:44 PM

Anyone know what's up with this? Hints for where to start looking?

[ Permlink | « Hide ]

Amar Kamat added a comment - 08/Jun/09 08:48 AM

I tried reproducing this error but I couldnt. Can anyone plz comment as to how to reproduce this bug. It never failed on my machine.

[ Permlink | « Hide ]

Ravi Gummadi added a comment - 09/Jun/09 04:51 AM

Amar, Please run the test few times. Seems to be failing once in a while with that error.

[ Permlink | « Hide ]

Amar Kamat added a comment - 11/Jun/09 09:19 AM

Looks like the issue it to do with

reuse of same mapred.system.dir and hadoop.log.dir
restart

[ Permlink | « Hide ]

Amar Kamat added a comment - 12/Jun/09 08:28 AM

Attaching a patch that deletes the mapred.system.dir and hadoop.log.history.location after every test. Result of test-patch
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

ant test passed on my box.

[ Permlink | « Hide ]

Ravi Gummadi added a comment - 16/Jun/09 06:27 AM

If this doesn't happen in real world, then fixing the testcase should be fine.
Ran the testcase few times with the patch and passed.
+1

[ Permlink | « Hide ]

Amar Kamat added a comment - 16/Jun/09 01:19 PM

Tested this patch by setting the jobtracker identifier to a fixed value and jobtracker start-time to a different value. With this patch it passes always.

[ Permlink | « Hide ]

Devaraj Das added a comment - 17/Jun/09 12:29 PM

I just committed this. Thanks, Amar!

[ Permlink | « Hide ]

Hudson added a comment - 17/Jun/09 07:19 PM

Integrated in Hadoop-trunk #869 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/869/)
. Fixes a testcase failure for TestJobHistory. Contributed by Amar Kamat.

[ Permlink | « Hide ]

Amar Kamat added a comment - 18/Jun/09 03:35 AM

Attaching a patch for branch 0.20.