[OOZIE-3170] Oozie Diagnostic Bundle tool fails with NPE due to missing service class

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.0.0b1
• Assignee: Jason Phelps
• Reporter: Jason Phelps
• Created: 26/Jan/18 15:47
• Updated: 20/Feb/18 14:42

Description

 

When I ran the below command after doing a clean build from the main branch

bin/oozie-diag-bundle-collector.sh -oozie http://jphelps-60-1.gce.cloudera.com:11000/oozie -output /tmp/jobs/

It will fail with an NPE. I apologize as I did not copy the client error, but the error in oozie.log is below:

2018-01-25 10:53:58,123 ERROR ShareLibService:517 - SERVER[] org.apache.oozie.service.ServiceException: E0104: Could not fully initialize service [org.apache.oozie.service.ShareLibService], Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib
org.apache.oozie.service.ServiceException: E0104: Could not fully initialize service [org.apache.oozie.service.ShareLibService], Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib
 at org.apache.oozie.service.ShareLibService.init(ShareLibService.java:144)
 at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
 at org.apache.oozie.service.Services.setService(Services.java:372)
 at org.apache.oozie.service.Services.loadServices(Services.java:304)
 at org.apache.oozie.service.Services.init(Services.java:212)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.initOozieServices(DiagBundleCollectorDriver.java:153)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.setHadoopConfig(DiagBundleCollectorDriver.java:135)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.run(DiagBundleCollectorDriver.java:56)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.main(DiagBundleCollectorDriver.java:52)
Caused by: java.lang.NullPointerException
 at org.apache.oozie.service.ShareLibService.cacheActionKeySharelibConfList(ShareLibService.java:878)
 at org.apache.oozie.service.ShareLibService.init(ShareLibService.java:132)
 ... 8 more

2018-01-25 10:53:58,130 INFO PartitionDependencyManagerService:520 - SERVER[] PartitionDependencyManagerService initialized. Dependency cache is org.apache.oozie.dependency.hcat.SimpleHCatDependencyCache

2018-01-25 10:53:58,131 FATAL Services:514 - SERVER[] Runtime Exception during Services Load. Check your list of {0} or {1}
java.lang.NullPointerException
 at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:81)
 at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:71)
 at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
 at org.apache.oozie.service.Services.setService(Services.java:372)
 at org.apache.oozie.service.Services.loadServices(Services.java:304)
 at org.apache.oozie.service.Services.init(Services.java:212)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.initOozieServices(DiagBundleCollectorDriver.java:153)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.setHadoopConfig(DiagBundleCollectorDriver.java:135)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.run(DiagBundleCollectorDriver.java:56)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.main(DiagBundleCollectorDriver.java:52)

2018-01-25 10:53:58,132 FATAL Services:514 - SERVER[] E0103: Could not load service classes, null
org.apache.oozie.service.ServiceException: E0103: Could not load service classes, null
 at org.apache.oozie.service.Services.loadServices(Services.java:309)
 at org.apache.oozie.service.Services.init(Services.java:212)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.initOozieServices(DiagBundleCollectorDriver.java:153)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.setHadoopConfig(DiagBundleCollectorDriver.java:135)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.run(DiagBundleCollectorDriver.java:56)
 at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.main(DiagBundleCollectorDriver.java:52)
Caused by: java.lang.NullPointerException
 at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:81)
 at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:71)
 at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
 at org.apache.oozie.service.Services.setService(Services.java:372)
 at org.apache.oozie.service.Services.loadServices(Services.java:304)
 ... 5 more
 

From my debugging, it looks like it needs the JobsConcurrencyService to run
https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/PartitionDependencyManagerService.java#L81 

purgeEnabled = Services.get().get(JobsConcurrencyService.class).isHighlyAvailableMode();

But this service is not loaded by the following:

https://github.com/apache/oozie/blob/master/tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCollectorDriver.java#L149

services.getConf()
 .set(Services.CONF_SERVICE_CLASSES, "org.apache.oozie.service.LiteWorkflowAppService,"
 + "org.apache.oozie.service.SchedulerService,"
 + "org.apache.oozie.service.HadoopAccessorService,"
 + "org.apache.oozie.service.ShareLibService");