Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.5.2, 0.6.0, 0.7.0, 0.6.1, 0.6.2, 0.6.3
-
None
-
IBM IOP 4.2. Only reproduced on Ranger 0.5.2 but the relevant code seems unchanged in newer versions.
Description
EmbeddedServiceDefsUtil have this method (I removed some log statements to make the code more clear)
public class EmbeddedServiceDefsUtil { ... private RangerServiceDef getOrCreateServiceDef(ServiceStore store, String serviceDefName) { .... RangerServiceDef ret = null; boolean createServiceDef = (CollectionUtils.isEmpty(supportedServiceDefs) || supportedServiceDefs.contains(serviceDefName)); try { ret = store.getServiceDefByName(serviceDefName); if(ret == null && createEmbeddedServiceDefs && createServiceDef) { ret = loadEmbeddedServiceDef(serviceDefName); if (ret.getId() != null) { store.setPopulateExistingBaseFields(true); ret = store.createServiceDef(ret); store.setPopulateExistingBaseFields(false); } else { ret = store.createServiceDef(ret); } LOG.info("created embedded service-def " + serviceDefName); } } catch(Exception excp) { LOG.fatal("EmbeddedServiceDefsUtil.getOrCreateServiceDef(): failed to load/create serviceType " + serviceDefName, excp); } ... return ret; } }
When creating a service def, the flag PopulateExistingBaseFields must be set to true, but it should be set to false afterwards.
If the statement ``` ret = store.createServiceDef(ret); ``` throws an exception, the flag is never reset. But since the catch statement do not rethrow the exception, null will be returned and Ranger will start normally.
Now, the flag PopulateExistingBaseFields are used by a lot of methods in ServiceDBStore
- public RangerServiceDef *createServiceDef*(RangerServiceDef serviceDef) throws Exception
- public RangerService *createService*(RangerService service) throws Exception
- public RangerService *updateService*(RangerService service) throws Exception
- public RangerPolicy *createPolicy*(RangerPolicy policy) throws Exception {
For createService and updateService, it will cause the method to select between svcServiceWithAssignedId and svcService. svcServiceWithAssignedId require that the incoming service have an ID. Services created via the web gui and via the ambari integration do NOT have a pre-selected ID. They cannot be created, and thus you cannot actually use Ranger.
Now, how can ```ret = store.createServiceDef(ret);``` fail? I do not know, but it happened on my system with this exception log entry
2016-12-06 09:34:25,850 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil (EmbeddedServiceDefsUtil.java:136) - ==> EmbeddedServiceDefsUtil.getOrCreateServiceDef(kms) 2016-12-06 09:34:25,850 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:934) - ==> ServiceDefDBStore.getServiceDefByName(kms) 2016-12-06 09:34:25,855 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:946) - == ServiceDefDBStore.getServiceDefByName(kms): null 2016-12-06 09:34:25,855 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil (EmbeddedServiceDefsUtil.java:168) - ==> EmbeddedServiceDefsUtil.loadEmbeddedServiceDef(kms) 2016-12-06 09:34:25,867 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil (EmbeddedServiceDefsUtil.java:182) - ==> EmbeddedServiceDefsUtil.loadEmbeddedServiceDef(kms) 2016-12-06 09:34:25,867 [kact-man-001.kact.sblokalnet-startStop-1] INFO org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil (EmbeddedServiceDefsUtil.java:146) - creating embedded service-def kms 2016-12-06 09:34:25,868 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:242) - ==> ServiceDefDBStore.createServiceDef(RangerServiceDef={id={7} guid={null} isEnabled={true} createdBy={null} updatedBy={null} createTime={null} updateTime={null} version={null} name={kms} implClass={org.apache.ranger.services.kms.RangerServiceKMS} label={KMS} description={KMS} rbKeyLabel={null} rbKeyDescription={null} c onfigs={RangerServiceConfigDef={itemId={provider} name={provider} type={string} subType={null} mandatory={true} defaultValue={null} validationRegEx={null} validationMessage={null} uiHint={null} label={KMS URL} description={null} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={username} name={username} type={string} subType={null} mandatory={true} defaultValue={null} validationRegEx={null} validationM essage={null} uiHint={null} label={Username} description={null} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={password} name={password} type={password} subType={null} mandatory={true} defaultValue={null} validationRegEx={null} validationMessage={null} uiHint={null} label={Password} description={null} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }} resources={RangerResourc eDef={itemId={1} name={keyname} type={string} level={10} parent={} mandatory={true} lookupSupported={true} recursiveSupported={false} excludesSupported={false} matcher={org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher} matcherOptions={{wildCard=true, ignoreCase=false}} validationRegEx={} validationMessage={} uiHint={} label={Key Name} description={Key Name} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }} acce ssTypes={RangerAccessTypeDef={itemId={1} name={create} label={Create} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={2} name={delete} label={Delete} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={3} name={rollover} label={Rollover} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={4} name={setkeymaterial} label={Set Key Material} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={5} name={get } label={Get} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={6} name={getkeys} label={Get Keys} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={7} name={getmetadata} label={Get Metadata} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={8} name={generateeek} label={Generate EEK} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={9} name={decrypteek} label={Decrypt EEK} rbKeyLabel={null} implie dGrants={} }} policyConditions={} contextEnrichers={} enums={} }) 2016-12-06 09:34:25,870 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.common.db.BaseDao (BaseDao.java:239) - Ignoring BaseDao.setIdentityInsert(). This should be executed if DB flavor is sqlserver. 2016-12-06 09:34:25,872 [kact-man-001.kact.sblokalnet-startStop-1] FATAL org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil (EmbeddedServiceDefsUtil.java:156) - EmbeddedServiceDefsUtil.getOrCreateServiceDef(): failed to load/create serviceType kms java.lang.NullPointerException at org.apache.ranger.service.RangerServiceDefServiceBase.mapViewToEntityBean(RangerServiceDefServiceBase.java:158) at org.apache.ranger.service.RangerServiceDefWithAssignedIdService.mapViewToEntityBean(RangerServiceDefWithAssignedIdService.java:31) at org.apache.ranger.service.RangerServiceDefWithAssignedIdService.mapViewToEntityBean(RangerServiceDefWithAssignedIdService.java:25) at org.apache.ranger.service.RangerBaseModelService.populateEntityBeanForCreate(RangerBaseModelService.java:180) at org.apache.ranger.service.RangerBaseModelService.preCreate(RangerBaseModelService.java:217) at org.apache.ranger.service.RangerBaseModelService.create(RangerBaseModelService.java:225) at org.apache.ranger.biz.ServiceDBStore.createServiceDef(ServiceDBStore.java:268) at org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil.getOrCreateServiceDef(EmbeddedServiceDefsUtil.java:149) at org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil.init(EmbeddedServiceDefsUtil.java:96) at org.apache.ranger.biz.ServiceDBStore$1.doInTransaction(ServiceDBStore.java:223) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130) at org.apache.ranger.biz.ServiceDBStore.initStore(ServiceDBStore.java:220) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299) at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1448) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:605) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:925) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:472) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2016-12-06 09:34:25,875 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil (EmbeddedServiceDefsUtil.java:160) - <== EmbeddedServiceDefsUtil.getOrCreateServiceDef(kms): RangerServiceDef={id={7} guid={null} isEnabled={true} createdBy={null} updatedBy={null} createTime={null} updateTime={null} version={1} name={kms} implClass={org.apache.ranger.services.kms.RangerServiceKMS} label={KMS} description={KMS} rbKeyLabel={null} rbKeyDescription={null} configs={RangerServiceConfigDef={itemId={provider} name={provider} type={string} subType={null} mandatory={true} defaultValue={null} validationRegEx={null} validationMessage={null} uiHint={null} label={KMS URL} description={null} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={username} name={username} type={string} subType={null} mandatory={true} defaultValue={null} validationRegEx={null} validationMessage={null} uiHint={null} label={Username} description={null} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={password} name={password} type={password} subType={null} mandatory={true} defaultValue={null} validationRegEx={null} validationMessage={null} uiHint={null} label={Password} description={null} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }} resources={RangerResourceDef={itemId={1} name={keyname} type={string} level={10} parent={} mandatory={true} lookupSupported={true} recursiveSupported={false} excludesSupported={false} matcher={org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher} matcherOptions={{wildCard=true, ignoreCase=false}} validationRegEx={} validationMessage={} uiHint={} label={Key Name} description={Key Name} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }} accessTypes={RangerAccessTypeDef={itemId={1} name={create} label={Create} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={2} name={delete} label={Delete} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={3} name={rollover} label={Rollover} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={4} name={setkeymaterial} label={Set Key Material} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={5} name={get} label={Get} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={6} name={getkeys} label={Get Keys} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={7} name={getmetadata} label={Get Metadata} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={8} name={generateeek} label={Generate EEK} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={9} name={decrypteek} label={Decrypt EEK} rbKeyLabel={null} impliedGrants={} }} policyConditions={} contextEnrichers={} enums={} }