[MAPREDUCE-194] Split Information errors when input data volumn is trivial

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed

Environment

CentOS 5,
hadoop-0.19.0

• Assignee: Unassigned
• Reporter: Leitao Guo
• Created: 12/Mar/09 08:53
• Updated: 21/Jul/14 21:54
• Resolved: 21/Jul/14 21:54

Description

The mapreduce input is a text file with only 8 lines ( filepath: /in_wc/pretty ), and we set "conf.setNumMapTasks(8)" in the program. I thought there will generate 8 maptasks, but actually, it generated 9 maptask. Counters of map tasks from the website show that, 0~7 maptask has "Map input records 1", and 8 maptask has "Map input records 0"

The following is map task list information:

task_200903121214_0029_m_000000 hdfs://guoleitao:9200/in_wc/pretty:0+4

task_200903121214_0029_m_000001 hdfs://guoleitao:9200/in_wc/pretty:4+4

task_200903121214_0029_m_000002 hdfs://guoleitao:9200/in_wc/pretty:8+4

task_200903121214_0029_m_000003 hdfs://guoleitao:9200/in_wc/pretty:12+4

task_200903121214_0029_m_000004 hdfs://guoleitao:9200/in_wc/pretty:16+4

task_200903121214_0029_m_000005 hdfs://guoleitao:9200/in_wc/pretty:20+4

task_200903121214_0029_m_000006 hdfs://guoleitao:9200/in_wc/pretty:24+4

task_200903121214_0029_m_000007 hdfs://guoleitao:9200/in_wc/pretty:28+4

task_200903121214_0029_m_000008 hdfs://guoleitao:9200/in_wc/pretty:32+4