[#MAPREDUCE-131] After HADOOP-5420, tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir

Hadoop Map/Reduce

After HADOOP-5420, tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir

Created: 12/Jun/09 12:59 PM Updated: 23/Jun/09 08:26 AM

Return to search

Component/s:

None

Affects Version/s:

None

Fix Version/s:

None

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	hadoop-6027-1.patch	2009-06-15 09:42 AM	Sreekanth Ramakrishnan	3 kB

Issue Links:

Duplicate

This issue is duplicated by:

~~MAPREDUCE-842~~ Per-job local data on the TaskTracker node should have right access-control

Resolution Date:

23/Jun/09 08:26 AM

Description

« Hide

~~HADOOP-5420~~ refactored code from taskcontroller.c -> check_tt_root() to configuration.c for reuse. While doing this, this bug got introduced because of which all tasks hang when a cluster is configured with multiple mapred local directories. This happens because of the latest code in check_tt_root() which never advances the mapred_local_dir pointer and gets stuck in an infinite loop.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

Sreekanth Ramakrishnan added a comment - 15/Jun/09 09:42 AM

Attaching a patch which fixes this issue:

Advancing the pointer in check_tt_root() function.
Fixing a bug in MiniMRCluster which previous didnt use numDir
Modified ClusterWithLinuxTaskController to always use 4 local directories.

[ Permlink | « Hide ]

Vinod K V added a comment - 23/Jun/09 08:26 AM

Incorporated in HADOOP-4491