Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3170

Oozie Diagnostic Bundle tool fails with NPE due to missing service class

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 5.0.0b1
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

       

      When I ran the below command after doing a clean build from the main branch

      bin/oozie-diag-bundle-collector.sh -oozie http://jphelps-60-1.gce.cloudera.com:11000/oozie -output /tmp/jobs/
      

      It will fail with an NPE. I apologize as I did not copy the client error, but the error in oozie.log is below:

      2018-01-25 10:53:58,123 ERROR ShareLibService:517 - SERVER[] org.apache.oozie.service.ServiceException: E0104: Could not fully initialize service [org.apache.oozie.service.ShareLibService], Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib
      org.apache.oozie.service.ServiceException: E0104: Could not fully initialize service [org.apache.oozie.service.ShareLibService], Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib
       at org.apache.oozie.service.ShareLibService.init(ShareLibService.java:144)
       at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
       at org.apache.oozie.service.Services.setService(Services.java:372)
       at org.apache.oozie.service.Services.loadServices(Services.java:304)
       at org.apache.oozie.service.Services.init(Services.java:212)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.initOozieServices(DiagBundleCollectorDriver.java:153)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.setHadoopConfig(DiagBundleCollectorDriver.java:135)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.run(DiagBundleCollectorDriver.java:56)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.main(DiagBundleCollectorDriver.java:52)
      Caused by: java.lang.NullPointerException
       at org.apache.oozie.service.ShareLibService.cacheActionKeySharelibConfList(ShareLibService.java:878)
       at org.apache.oozie.service.ShareLibService.init(ShareLibService.java:132)
       ... 8 more
      
      2018-01-25 10:53:58,130 INFO PartitionDependencyManagerService:520 - SERVER[] PartitionDependencyManagerService initialized. Dependency cache is org.apache.oozie.dependency.hcat.SimpleHCatDependencyCache
      
      2018-01-25 10:53:58,131 FATAL Services:514 - SERVER[] Runtime Exception during Services Load. Check your list of {0} or {1}
      java.lang.NullPointerException
       at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:81)
       at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:71)
       at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
       at org.apache.oozie.service.Services.setService(Services.java:372)
       at org.apache.oozie.service.Services.loadServices(Services.java:304)
       at org.apache.oozie.service.Services.init(Services.java:212)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.initOozieServices(DiagBundleCollectorDriver.java:153)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.setHadoopConfig(DiagBundleCollectorDriver.java:135)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.run(DiagBundleCollectorDriver.java:56)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.main(DiagBundleCollectorDriver.java:52)
      
      2018-01-25 10:53:58,132 FATAL Services:514 - SERVER[] E0103: Could not load service classes, null
      org.apache.oozie.service.ServiceException: E0103: Could not load service classes, null
       at org.apache.oozie.service.Services.loadServices(Services.java:309)
       at org.apache.oozie.service.Services.init(Services.java:212)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.initOozieServices(DiagBundleCollectorDriver.java:153)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.setHadoopConfig(DiagBundleCollectorDriver.java:135)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.run(DiagBundleCollectorDriver.java:56)
       at org.apache.oozie.tools.diag.DiagBundleCollectorDriver.main(DiagBundleCollectorDriver.java:52)
      Caused by: java.lang.NullPointerException
       at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:81)
       at org.apache.oozie.service.PartitionDependencyManagerService.init(PartitionDependencyManagerService.java:71)
       at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
       at org.apache.oozie.service.Services.setService(Services.java:372)
       at org.apache.oozie.service.Services.loadServices(Services.java:304)
       ... 5 more
       
      

      From my debugging, it looks like it needs the JobsConcurrencyService to run
      https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/PartitionDependencyManagerService.java#L81 

      purgeEnabled = Services.get().get(JobsConcurrencyService.class).isHighlyAvailableMode();

      But this service is not loaded by the following:

      https://github.com/apache/oozie/blob/master/tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCollectorDriver.java#L149

      services.getConf()
       .set(Services.CONF_SERVICE_CLASSES, "org.apache.oozie.service.LiteWorkflowAppService,"
       + "org.apache.oozie.service.SchedulerService,"
       + "org.apache.oozie.service.HadoopAccessorService,"
       + "org.apache.oozie.service.ShareLibService");

        Attachments

        1. OOZIE-3170-1.patch
          0.9 kB
          Jason Phelps
        2. OOZIE-3170-2.patch
          4 kB
          Jason Phelps

          Activity

            People

            • Assignee:
              jphelps Jason Phelps
              Reporter:
              jphelps Jason Phelps
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: