[GIRAPH-822] java.lang.ClassCastException: org.apache.giraph.conf.GiraphConfiguration cannot be cast to org.apache.hadoop.mapred.JobConf

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.0.0
• Component/s: mapreduce

Environment

CentOS 6.5

• Assignee: Unassigned
• Reporter: John Yost
• Created: 08/Jan/14 15:29
• Updated: 08/Jan/14 16:57

Description

Hi Everyone,

I've run into this really weird error with the following stacktrace when I attempt to execute a Giraph job via a custom Tool class:

Exception in thread "main" java.lang.ClassCastException: org.apache.giraph.conf.GiraphConfiguration cannot be cast to org.apache.hadoop.mapred.JobConf
at org.apache.hadoop.mapreduce.Job$1.run(Job.java:513)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:511)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:499)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at com.prototype.analytics.TwitterSpringRunner.run(TwitterSpringRunner.java:58)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at com.prototype.analytics.TwitterSpringRunner.main(TwitterSpringRunner.java:78)

UPDATE:
When I revert back to using GiraphRunner, the job completes successfully, so I must have dorked up something along the way within my Tool class.

Also, this ClassCastException is thrown when I make the following method invocation:
job.getInternalJob().waitForCompletion(false);

When I remove this, the job starts, but no tasks are executed

Here's the run method implementation:

@Override
public int run(String[] parameters) throws Exception

{ String jobName = parameters[0]; String zConfig = parameters[1]; String inputPath = parameters[2]; String outputPath = parameters[3]; String seedVertex = parameters[4]; int numWorkers = Integer.parseInt(parameters[5]); GiraphConfiguration gConf = new GiraphConfiguration(getConf()); gConf.set("mapred.job.tracker","localhost:9001"); gConf.set("fs.default.name","hdfs://localhost:9000"); gConf.setVertexClass(com.johnyost.analytics.io.TwitterVertex.class); gConf.setMasterComputeClass(TwitterCompute.class); gConf.setZooKeeperConfiguration(zConfig); gConf.setWorkerConfiguration(numWorkers,numWorkers,100.0f); gConf.set("seedVertex",seedVertex); gConf.setVertexInputFormatClass((Class<? extends VertexInputFormat>) JsonTwitterInputFormat.class); gConf.setVertexOutputFormatClass((Class<? extends VertexOutputFormat>) org.apache.giraph.io.formats.IdWithValueTextOutputFormat.class); GiraphJob job = new GiraphJob(gConf,jobName); FileInputFormat.setInputPaths(job.getInternalJob(),new Path(inputPath)); FileOutputFormat.setOutputPath(job.getInternalJob(),new Path(outputPath)); GiraphFileInputFormat.setVertexInputPath(gConf,new Path(inputPath)); log.info(job.getInternalJob().getConfiguration().get("mapred.job.tracker")); log.info(job.getInternalJob().getConfiguration().get("fs.default.name")); job.getInternalJob().waitForCompletion(false); return job.run(true) ? 0 : -1; }

Any ideas as to why my custom Tool class is failing while GiraphRunner is not? This was working, but I must have changed something yesterday (not sure what it is, was not in this class) and now I cannot get the Giraph job to execute.