Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.9.2
-
None
-
None
-
None
Description
Running Pig scripts through Grunt works without hiccup but I have a utility program that runs commands through PigServer. When running that way, I run into two problems.
The first, which I can hack around, is that I need to pass in to PigServer the hbase configuration. When I do this with ConfigurationUtil.toProperties(hbaseConfig), the new properties get passed in-not the deprecated ones. Later on down the road, Pig looks for fs.default.name which is the deprecated property for fs.defaultfs and throws [1] The hack is to back map the fs.default.name property to the value of fs.defaultfs after calling ConfigurationUtil.toProperties();
That brings me to the second problem which I haven't been able to hack around - When PigServer runs, it throws the exception[2]. Poking through the code, there is no longer an org.apache.hadoop.mapred.jobcontrol.JobControl.addJob() method distributed with hadoop 0.23-only an org.apache.hadoop.mapred.jobcontrol.JobControl.addJobs() method-which also appears to be deprecated.
Again, Grunt works fine-just PigServer shows these problems. Has anyone resolved this? Or should I log a bug?
Thanks!
[1]
java.lang.NullPointerException
at java.net.URI$Parser.parse(URI.java:3003)
at java.net.URI.<init>(URI.java:578)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.setHdfsServers(QueryParser.java:485)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.LoadClause(QueryParser.java:1618)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1366)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:1013)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:800)
at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1612)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1562)
at org.apache.pig.PigServer.registerQuery(PigServer.java:534)
at org.apache.pig.PigServer.registerQuery(PigServer.java:558)
at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:227)
[2]
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Lja
va/lang/String;
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:258)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
at org.apache.pig.PigServer.storeEx(PigServer.java:885)
at org.apache.pig.PigServer.store(PigServer.java:827)
at com.cerner.kepler.pig.console.KeyDump.dumpKeys(KeyDump.java:231)