[HAMA-862] Error partitioning the input path

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.3
• Fix Version/s: 0.7.0
• Component/s: bsp core
• Assignee: Edward J. Yoon
• Reporter: Edward J. Yoon
• Created: 27/Jan/14 08:13
• Updated: 04/Feb/14 01:20
• Resolved: 03/Feb/14 01:44

Description

It seems error occurs when block files num > desired tasks num.

edward@edward-VirtualBox:~/workspace/hama-0.6.3$ bin/hama jar examples/target/hama-examples-0.6.3.jar gen fastgen 7000000 3 trunk/7000000-3 3
..

edward@edward-VirtualBox:~/workspace/hama-trunk$ ~/hadoop-1.2.1/bin/hadoop dfs -ls /user/edward/trunk/7000000-3
Found 3 items
-rw-r--r--   3 edward supergroup  101439512 2014-01-27 17:06 /user/edward/trunk/7000000-3/part-00000
-rw-r--r--   3 edward supergroup  102565269 2014-01-27 17:06 /user/edward/trunk/7000000-3/part-00001
-rw-r--r--   3 edward supergroup  102561984 2014-01-27 17:06 /user/edward/trunk/7000000-3/part-00002

edward@edward-VirtualBox:~/workspace/hama-0.6.3$ bin/hama jar examples/target/hama-examples-0.6.3.jar pagerank trunk/7000000-3 /tmp/result 3
14/01/27 17:07:43 INFO bsp.FileInputFormat: Total input paths to process : 3
14/01/27 17:07:43 INFO bsp.FileInputFormat: Total input paths to process : 3
14/01/27 17:07:43 INFO bsp.BSPJobClient: Running job: job_201401271704_0007
14/01/27 17:07:43 INFO bsp.BSPJobClient: Job failed.
14/01/27 17:07:43 ERROR bsp.BSPJobClient: Error partitioning the input path.
java.io.IOException: Runtime partition failed for the job.
	at org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465)
	at org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333)
	at org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293)
	at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:228)
	at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203)
	at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:235)
	at org.apache.hama.examples.PageRank.main(PageRank.java:164)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hama.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
	at org.apache.hama.util.ProgramDriver.driver(ProgramDriver.java:139)
	at org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hama.util.RunJar.main(RunJar.java:146)