Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Duplicate
-
5.1.0
-
None
-
None
-
None
Description
We can no longer submit a coordinator job after upgrading from 5.0.0 to 5.1.0. While we investigate this (there is an email in the oozie user mailing list), I've been looking at the oozie code trying to determine why the error message has "null" in it.
Error: E0307 : E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
We are submitting a coordinator job so it specifies oozie.coord.application.path.
Here is a sample submission:
sudo -u oozie oozie job -oozie http://hbase.qa.local:11000/oozie/ -config /var/lib/cmx/jobs/workspaceDimToVertica/coordinator.properties -run -D nameNode=hdfs://hbase.qa.local:8020 -D jobTracker=hbase.qa.local:8032 -D frequency=240 -D startTime=2018-03-28T00:00Z -D endTime=2020-01-01:00Z -debug
Auth type : null
POST http://hbase.qa.local:11000/oozie/v2/jobs?action=start
<?xml version="1.0" encoding="UTF-8" standalone="no"?><configuration>
<property><name>nameNode</name><value>hdfs://hbase.qa.local:8020</value></property>
<property><name>oozie.use.system.libpath</name><value>true</value></property>
<property><name>startTime</name><value>2018-03-28T00:00Z</value></property>
<property><name>queueName</name><value>oozie</value></property>
<property><name>jobTracker</name><value>hbase.qa.local:8032</value></property>
<property><name>oozie.libpath</name><value>hdfs://hbase.qa.local:8020/user/oozie/cmx</value></property>
<property><name>endTime</name><value>2020-01-01:00Z</value></property>
<property><name>applicationConfigFile</name><value>hdfs://hbase.qa.local:8020/usr/share/cmx/conf/app.properties,hdfs://hbase.qa.local:8020/usr/share/cmx/conf/crypto.properties</value></property>
<property><name>frequency</name><value>240</value></property>
<property><name>user.name</name><value>oozie</value></property>
<property><name>oozie.coord.application.path</name><value>hdfs://hbase.qa.local:8020/user/oozie/reporting/workspaceDimToVertica/</value></property>
</configuration>
V1JobsServlet.java:175 creates an error message using the appPath from oozie.wf.application.path. But, as you will see the method also checks other appPath. Building the error message should consider these other appPaths as it would help troubleshoot errors such as these.
Here is the server-side the stack trace.
2019-03-28 20:00:56,101 WARN V1JobsServlet:523 - SERVER[ip-172-23-113-207.ec2.internal] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] URL[POST http://hbase.qa.local:11000/oozie/v2/jobs] error[E0307], E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
org.apache.oozie.servlet.XServletException: E0307: Runtime error [App directory [null] does not exist and app definition cannot be created because of missing config value [oozie.jobs.api.generated.xml]]
at org.apache.oozie.servlet.V1JobsServlet.checkAndWriteApplicationXMLToHDFS(V1JobsServlet.java:175)
at org.apache.oozie.servlet.BaseJobsServlet.doPost(BaseJobsServlet.java:111)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:305)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:171)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:176)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.oozie.servlet.HostnameFilter.doFilter(HostnameFilter.java:86)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Attachments
Issue Links
- duplicates
-
-
- Closed
-