Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.9.0
-
None
Description
A customer hit a rebind error. Brooklyn failed to instantiate the entity because the driver's constructor called SshMachineLocation.getOsDetails, which tried to call the vCloudDirector API. This call failed because the VM no longer exists. The exception was propagated, causing the entity to fail to be re-created.
We should avoid propagating the exception, e.g. by returning an OsDetails saying "unknown".
Even better, we shouldn't be calling the cloud api at this point. But the call to getOsDetails is happening from the customer code.
Should we store the OsDetails in the machine persisted state, so we don't need to look it up again?
2016-07-11 12:46:13,258 ERROR o.a.b.l.common.BasicLauncher [main]: Subsystem for persistence had startup error (continuing with startup): org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: Failure rebinding; 2 errors including: problem rebinding entity DKtbsydb (JenkinsNodeImpl{id=DKtbsydb}): InvalidCacheLoadException: CacheLoader returned null for key urn:vcloud:vm:43ebf981-3900-491c-bfd1-68ca1ab3a9ea.
org.apache.brooklyn.util.exceptions.PropagatedRuntimeException:
at org.apache.brooklyn.util.exceptions.Exceptions.propagate(Exceptions.java:128) ~[brooklyn-utils-common-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl.rebind(RebindManagerImpl.java:513) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl.promoteToMaster(HighAvailabilityManagerImpl.java:836) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl.checkMaster(HighAvailabilityManagerImpl.java:810) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl.publishAndCheck(HighAvailabilityManagerImpl.java:592) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl.changeMode(HighAvailabilityManagerImpl.java:315) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.ha.HighAvailabilityManagerImpl.start(HighAvailabilityManagerImpl.java:255) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.launcher.common.BasicLauncher.startPersistence(BasicLauncher.java:642) [brooklyn-launcher-common-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.launcher.common.BasicLauncher.handlePersistence(BasicLauncher.java:506) [brooklyn-launcher-common-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.launcher.common.BasicLauncher.start(BasicLauncher.java:424) [brooklyn-launcher-common-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.cli.Main$LaunchCommand.call(Main.java:461) [brooklyn-cli-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.cli.Main$LaunchCommand.call(Main.java:196) [brooklyn-cli-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.cli.AbstractMain.execCli(AbstractMain.java:252) [brooklyn-cli-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.cli.AbstractMain.execCli(AbstractMain.java:244) [brooklyn-cli-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at io.cloudsoft.amp.Main.main(Main.java:36) [amp-launcher-3.1.1.jar:na]
Caused by: java.util.concurrent.ExecutionException: org.apache.brooklyn.util.exceptions.CompoundRuntimeException: Failure rebinding; 2 errors including: problem rebinding entity DKtbsydb (JenkinsNodeImpl{id=DKtbsydb}): InvalidCacheLoadException: CacheLoader returned null for key urn:vcloud:vm:43ebf981-3900-491c-bfd1-68ca1ab3a9ea.
at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.7.0_101]
at java.util.concurrent.FutureTask.get(FutureTask.java:188) ~[na:1.7.0_101]
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63) ~[guava-17.0.jar:na]
at org.apache.brooklyn.util.core.task.BasicTask.get(BasicTask.java:361) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl.rebind(RebindManagerImpl.java:511) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
... 13 common frames omitted
org.apache.brooklyn.util.exceptions.CompoundRuntimeException: Failure rebinding; 2 errors including: problem rebinding entity DKtbsydb (JenkinsNodeImpl{id=DKtbsydb}): InvalidCacheLoadException: CacheLoader returned null for key urn:vcloud:vm:43ebf981-3900-491c-bfd1-68ca1ab3a9ea.
at org.apache.brooklyn.util.exceptions.Exceptions.create(Exceptions.java:393) ~[brooklyn-utils-common-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindExceptionHandlerImpl.onDoneImpl(RebindExceptionHandlerImpl.java:497) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindExceptionHandlerImpl.onDone(RebindExceptionHandlerImpl.java:413) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindIteration.run(RebindIteration.java:267) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl.rebindImpl(RebindManagerImpl.java:558) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl$3.call(RebindManagerImpl.java:508) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindManagerImpl$3.call(RebindManagerImpl.java:506) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_101]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_101]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_101]
at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_101]
Caused by: java.lang.IllegalStateException: problem rebinding entity DKtbsydb (JenkinsNodeImpl{id=DKtbsydb})
at org.apache.brooklyn.core.mgmt.rebind.RebindExceptionHandlerImpl.onRebindFailed(RebindExceptionHandlerImpl.java:304) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindIteration.reconstructEverything(RebindIteration.java:625) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindIteration.doRun(RebindIteration.java:242) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.InitialFullRebindIteration.doRun(InitialFullRebindIteration.java:69) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindIteration.run(RebindIteration.java:265) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
... 8 common frames omitted
Caused by: org.apache.brooklyn.util.exceptions.PropagatedRuntimeException:
at org.apache.brooklyn.util.exceptions.Exceptions.propagate(Exceptions.java:128) ~[brooklyn-utils-common-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.entity.drivers.ReflectiveEntityDriverFactory.build(ReflectiveEntityDriverFactory.java:266) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.entity.drivers.BasicEntityDriverManager.build(BasicEntityDriverManager.java:53) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.entity.software.base.SoftwareProcessImpl.newDriver(SoftwareProcessImpl.java:126) ~[brooklyn-software-base-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.entity.software.base.SoftwareProcessImpl.doInitDriver(SoftwareProcessImpl.java:539) ~[brooklyn-software-base-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.entity.software.base.SoftwareProcessImpl.initDriver(SoftwareProcessImpl.java:519) ~[brooklyn-software-base-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.entity.software.base.SoftwareProcessImpl.rebind(SoftwareProcessImpl.java:458) ~[brooklyn-software-base-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.AbstractBrooklynObjectRebindSupport.reconstruct(AbstractBrooklynObjectRebindSupport.java:69) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.core.mgmt.rebind.RebindIteration.reconstructEverything(RebindIteration.java:623) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
... 11 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_101]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_101]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_101]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_101]
at org.apache.brooklyn.core.entity.drivers.ReflectiveEntityDriverFactory.build(ReflectiveEntityDriverFactory.java:263) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
... 18 common frames omitted
Caused by: com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key urn:vcloud:vm:43ebf981-3900-491c-bfd1-68ca1ab3a9ea.
at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348) ~[guava-17.0.jar:na]
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2318) ~[guava-17.0.jar:na]
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280) ~[guava-17.0.jar:na]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195) ~[guava-17.0.jar:na]
at com.google.common.cache.LocalCache.get(LocalCache.java:3934) ~[guava-17.0.jar:na]
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938) ~[guava-17.0.jar:na]
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821) ~[guava-17.0.jar:na]
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4827) ~[guava-17.0.jar:na]
at org.jclouds.vcloud.director.v1_5.functions.URNToHref.apply(URNToHref.java:49) ~[vcloud-director-1.9.2-cloudsoft.20160428.1116.jar:1.9.2-cloudsoft.20160428.1116]
at org.jclouds.vcloud.director.v1_5.functions.URNToHref.apply(URNToHref.java:37) ~[vcloud-director-1.9.2-cloudsoft.20160428.1116.jar:1.9.2-cloudsoft.20160428.1116]
at org.jclouds.rest.internal.RestAnnotationProcessor.getEndpointInParametersOrNull(RestAnnotationProcessor.java:495) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.RestAnnotationProcessor.getEndpointFor(RestAnnotationProcessor.java:518) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.RestAnnotationProcessor.findEndpoint(RestAnnotationProcessor.java:370) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:192) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:129) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.InvokeHttpMethod.toCommand(InvokeHttpMethod.java:188) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:84) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44) ~[jclouds-core-1.9.2.jar:1.9.2]
at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117) ~[jclouds-core-1.9.2.jar:1.9.2]
at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87) ~[guava-17.0.jar:na]
at com.sun.proxy.$Proxy201.get(Unknown Source) ~[na:na]
at org.jclouds.vcloud.director.v1_5.compute.strategy.VCloudDirectorComputeServiceAdapter.getNode(VCloudDirectorComputeServiceAdapter.java:597) ~[vcloud-director-1.9.2-cloudsoft.20160428.1116.jar:1.9.2-cloudsoft.20160428.1116]
at org.jclouds.vcloud.director.v1_5.compute.strategy.VCloudDirectorComputeServiceAdapter.getNode(VCloudDirectorComputeServiceAdapter.java:106) ~[vcloud-director-1.9.2-cloudsoft.20160428.1116.jar:1.9.2-cloudsoft.20160428.1116]
at org.jclouds.compute.strategy.impl.AdaptingComputeServiceStrategies.getNode(AdaptingComputeServiceStrategies.java:137) ~[jclouds-compute-1.9.2.jar:1.9.2]
at org.jclouds.compute.internal.BaseComputeService.getNodeMetadata(BaseComputeService.java:402) ~[jclouds-compute-1.9.2.jar:1.9.2]
at org.apache.brooklyn.location.jclouds.JcloudsSshMachineLocation.getOptionalNode(JcloudsSshMachineLocation.java:219) ~[brooklyn-locations-jclouds-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.location.jclouds.JcloudsSshMachineLocation.getOptionalOperatingSystem(JcloudsSshMachineLocation.java:498) ~[brooklyn-locations-jclouds-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.location.jclouds.JcloudsSshMachineLocation.inferMachineDetails(JcloudsSshMachineLocation.java:522) ~[brooklyn-locations-jclouds-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.location.ssh.SshMachineLocation.getMachineDetails(SshMachineLocation.java:1056) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at org.apache.brooklyn.location.ssh.SshMachineLocation.getOsDetails(SshMachineLocation.java:1037) ~[brooklyn-core-0.9.1-20160429.1930.jar:0.9.1-20160429.1930]
at com.acme.JenkinsSshDriver.<init>(JenkinsSshDriver.java:53) ~[acme-amp-jenkins-1.3.1-Final.jar:1.3-Final]
... 23 common frames omitted
Attachments
Issue Links
- links to
