Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Cannot Reproduce
-
None
-
None
-
None
-
None
Description
conf.set("mapred.output.committer.class", NullOutputCommitter.class.getName());
We've localized the hive jar, but this still causes a class not found exception.
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.ql.exec.NullOutputCommitter not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1657)
at org.apache.hadoop.mapred.MRVertexOutputCommitter.getOutputCommitter(MRVertexOutputCommitter.java:87)
at org.apache.hadoop.mapred.MRVertexOutputCommitter.init(MRVertexOutputCommitter.java:134)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:858)
at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:790)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:382)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:629)
at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:115)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:956)
at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:948)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:130)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.ql.exec.NullOutputCommitter not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1625)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1649)
... 15 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hive.ql.exec.NullOutputCommitter not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1531)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1623)
... 16 more