[HAMA-784] Input format class type is not considered when given multiple input paths

• Type: Bug
• Status: Open
• Priority: Trivial
• Resolution: Unresolved
• Component/s: bsp core
• Labels: inputformat
• Assignee: Unassigned
• Reporter: Ikhtiyor Ahmedov
• Created: 25/Jul/13 06:04
• Updated: 25/Jul/13 06:21

Description

Example code which fails:

KeyValueTextInputFormat.addInputPaths(job, "/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");

Exception:

java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to org.apache.hadoop.io.LongWritable
	at org.apache.hama.bsp.LineRecordReader.next(LineRecordReader.java:1)
	at org.apache.hama.bsp.TrackedRecordReader.moveToNext(TrackedRecordReader.java:63)
	at org.apache.hama.bsp.TrackedRecordReader.next(TrackedRecordReader.java:49)
	at org.apache.hama.bsp.BSPPeerImpl.readNext(BSPPeerImpl.java:598)
	at org.apache.hama.examples.OnlineCF$OnlineCFBSP.bsp(OnlineCF.java:146)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
	at org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:1)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)

Reason: below is part of implemented function of abstract class which is not overridden by extended classes (unfortunately static method)

FileInputFormat.java

public static void addInputPaths(BSPJob conf, String commaSeparatedPaths) {
  for (String str : getPathStrings(commaSeparatedPaths)) {
    addInputPath(conf, new Path(str));
  }
}

Quick fix:
anyway set input format

job.setInputFormat(KeyValueTextInputFormat.class);//set explicitly
KeyValueTextInputFormat.addInputPaths(job, "/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");

or add modification to the documentation in link (http://wiki.apache.org/hama/BSPModel), so that new developers will explicitly set input format even they use static methods of InputFormat.