Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-822

java.lang.ClassCastException: org.apache.giraph.conf.GiraphConfiguration cannot be cast to org.apache.hadoop.mapred.JobConf

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: mapreduce
    • Labels:
      None
    • Environment:

      CentOS 6.5

      Description

      Hi Everyone,

      I've run into this really weird error with the following stacktrace when I attempt to execute a Giraph job via a custom Tool class:

      Exception in thread "main" java.lang.ClassCastException: org.apache.giraph.conf.GiraphConfiguration cannot be cast to org.apache.hadoop.mapred.JobConf
      at org.apache.hadoop.mapreduce.Job$1.run(Job.java:513)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
      at org.apache.hadoop.mapreduce.Job.connect(Job.java:511)
      at org.apache.hadoop.mapreduce.Job.submit(Job.java:499)
      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
      at com.prototype.analytics.TwitterSpringRunner.run(TwitterSpringRunner.java:58)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at com.prototype.analytics.TwitterSpringRunner.main(TwitterSpringRunner.java:78)

      UPDATE:
      When I revert back to using GiraphRunner, the job completes successfully, so I must have dorked up something along the way within my Tool class.

      Also, this ClassCastException is thrown when I make the following method invocation:
      job.getInternalJob().waitForCompletion(false);

      When I remove this, the job starts, but no tasks are executed

      Here's the run method implementation:

      @Override
      public int run(String[] parameters) throws Exception

      { String jobName = parameters[0]; String zConfig = parameters[1]; String inputPath = parameters[2]; String outputPath = parameters[3]; String seedVertex = parameters[4]; int numWorkers = Integer.parseInt(parameters[5]); GiraphConfiguration gConf = new GiraphConfiguration(getConf()); gConf.set("mapred.job.tracker","localhost:9001"); gConf.set("fs.default.name","hdfs://localhost:9000"); gConf.setVertexClass(com.johnyost.analytics.io.TwitterVertex.class); gConf.setMasterComputeClass(TwitterCompute.class); gConf.setZooKeeperConfiguration(zConfig); gConf.setWorkerConfiguration(numWorkers,numWorkers,100.0f); gConf.set("seedVertex",seedVertex); gConf.setVertexInputFormatClass((Class<? extends VertexInputFormat>) JsonTwitterInputFormat.class); gConf.setVertexOutputFormatClass((Class<? extends VertexOutputFormat>) org.apache.giraph.io.formats.IdWithValueTextOutputFormat.class); GiraphJob job = new GiraphJob(gConf,jobName); FileInputFormat.setInputPaths(job.getInternalJob(),new Path(inputPath)); FileOutputFormat.setOutputPath(job.getInternalJob(),new Path(outputPath)); GiraphFileInputFormat.setVertexInputPath(gConf,new Path(inputPath)); log.info(job.getInternalJob().getConfiguration().get("mapred.job.tracker")); log.info(job.getInternalJob().getConfiguration().get("fs.default.name")); job.getInternalJob().waitForCompletion(false); return job.run(true) ? 0 : -1; }

      Any ideas as to why my custom Tool class is failing while GiraphRunner is not? This was working, but I must have changed something yesterday (not sure what it is, was not in this class) and now I cannot get the Giraph job to execute.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              hokiegeek2 John Yost
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: