Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-20937

Ambari Upgrade to 2.5 fails due to custom service; rpm upgrade does not preserve the custom service directory

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0
    • Fix Version/s: 2.5.1
    • Component/s: ambari-upgrade
    • Labels:
      None

      Description

      We have a 2.4.0.0 Ambari cluster, with HDP 2.5 and the custom service definition for CDAP installed (source here: https://github.com/caskdata/cdap-ambari-service). When attempting to upgrade to Ambari 2.5.0.3, the "ambari-server upgrade" command to upgrade the db schema fails, apparently due to the existence of the custom service. Is this expected, and is there a workaround?

      Steps taken:

      1. stop all cluster services
      2. stop ambari 2.4 server and agents
      3. yum upgrade the ambari packages
      4. run the schema upgrade as below:

      # ambari-server upgrade
      Using python  /usr/bin/python
      Upgrading ambari-server
      INFO: Upgrade Ambari Server
      INFO: Updating Ambari Server properties in ambari.properties ...
      INFO: Updating Ambari Server properties in ambari-env.sh ...
      WARNING: Original file ambari-env.sh kept
      INFO: Fixing database objects owner
      Ambari Server configured for Embedded Postgres. Confirm you have made a backup of the Ambari Server database [y/n] (y)? y
      INFO: Upgrading database schema
      INFO: Return code from schema upgrade command, retcode = 1
      ERROR: Error executing schema upgrade, please check the server logs.
      ERROR: Error output from schema upgrade command:
      ERROR: Exception in thread "main" org.apache.ambari.server.AmbariException: Guice provision errors:
      
      1) Error injecting method, java.lang.RuntimeException: Trying to create a ServiceComponent not recognized in stack info, clusterName=lamb-00022663, serviceName=CDAP, componentName=CDAP_CLI, stackInfo=HDP-2.5
        at org.apache.ambari.server.state.cluster.ClustersImpl.loadClustersAndHosts(ClustersImpl.java:186)
        at org.apache.ambari.server.state.cluster.ClustersImpl.class(ClustersImpl.java:102)
        while locating org.apache.ambari.server.state.cluster.ClustersImpl
        while locating org.apache.ambari.server.state.Clusters
          for field at org.apache.ambari.server.actionmanager.ActionDBAccessorImpl.clusters(ActionDBAccessorImpl.java:84)
        at org.apache.ambari.server.actionmanager.ActionDBAccessorImpl.class(ActionDBAccessorImpl.java:84)
        while locating org.apache.ambari.server.actionmanager.ActionDBAccessorImpl
        while locating org.apache.ambari.server.actionmanager.ActionDBAccessor
          for parameter 0 at org.apache.ambari.server.actionmanager.ActionManager.<init>(ActionManager.java:63)
        at org.apache.ambari.server.actionmanager.ActionManager.class(ActionManager.java:46)
        while locating org.apache.ambari.server.actionmanager.ActionManager
          for parameter 0 at org.apache.ambari.server.controller.AmbariManagementControllerImpl.<init>(AmbariManagementControllerImpl.java:356)
        at org.apache.ambari.server.controller.AmbariManagementControllerImpl.class(AmbariManagementControllerImpl.java:226)
        while locating org.apache.ambari.server.controller.AmbariManagementControllerImpl
        while locating org.apache.ambari.server.controller.AmbariManagementController
      
      1 error
      	at org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:240)
      	at org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:430)
      Caused by: com.google.inject.ProvisionException: Guice provision errors:
      
      1) Error injecting method, java.lang.RuntimeException: Trying to create a ServiceComponent not recognized in stack info, clusterName=lamb-00022663, serviceName=CDAP, componentName=CDAP_CLI, stackInfo=HDP-2.5
        at org.apache.ambari.server.state.cluster.ClustersImpl.loadClustersAndHosts(ClustersImpl.java:186)
        at org.apache.ambari.server.state.cluster.ClustersImpl.class(ClustersImpl.java:102)
        while locating org.apache.ambari.server.state.cluster.ClustersImpl
        while locating org.apache.ambari.server.state.Clusters
          for field at org.apache.ambari.server.actionmanager.ActionDBAccessorImpl.clusters(ActionDBAccessorImpl.java:84)
        at org.apache.ambari.server.actionmanager.ActionDBAccessorImpl.class(ActionDBAccessorImpl.java:84)
        while locating org.apache.ambari.server.actionmanager.ActionDBAccessorImpl
        while locating org.apache.ambari.server.actionmanager.ActionDBAccessor
          for parameter 0 at org.apache.ambari.server.actionmanager.ActionManager.<init>(ActionManager.java:63)
        at org.apache.ambari.server.actionmanager.ActionManager.class(ActionManager.java:46)
        while locating org.apache.ambari.server.actionmanager.ActionManager
          for parameter 0 at org.apache.ambari.server.controller.AmbariManagementControllerImpl.<init>(AmbariManagementControllerImpl.java:356)
        at org.apache.ambari.server.controller.AmbariManagementControllerImpl.class(AmbariManagementControllerImpl.java:226)
        while locating org.apache.ambari.server.controller.AmbariManagementControllerImpl
        while locating org.apache.ambari.server.controller.AmbariManagementController
      
      1 error
      	at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
      	at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
      	at org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.updateWidgetDefinitionsForService(AbstractUpgradeCatalog.java:1060)
      	at org.apache.ambari.server.upgrade.UpgradeCatalog2402.updateKafkaWidgetDefinition(UpgradeCatalog2402.java:204)
      	at org.apache.ambari.server.upgrade.UpgradeCatalog2402.executeDMLUpdates(UpgradeCatalog2402.java:190)
      	at org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeData(AbstractUpgradeCatalog.java:943)
      	at org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeDMLUpdates(SchemaUpgradeHelper.java:237)
      	... 1 more
      Caused by: java.lang.RuntimeException: Trying to create a ServiceComponent not recognized in stack info, clusterName=lamb-00022663, serviceName=CDAP, componentName=CDAP_CLI, stackInfo=HDP-2.5
      	at org.apache.ambari.server.state.ServiceComponentImpl.updateComponentInfo(ServiceComponentImpl.java:134)
      	at org.apache.ambari.server.state.ServiceComponentImpl.<init>(ServiceComponentImpl.java:164)
      	at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
      	at com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:260)
      	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
      	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
      	at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      	at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
      	at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)
      	at com.sun.proxy.$Proxy19.createExisting(Unknown Source)
      	at org.apache.ambari.server.state.ServiceImpl.<init>(ServiceImpl.java:169)
      	at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
      	at com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:260)
      	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
      	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
      	at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      	at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
      	at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)
      	at com.sun.proxy.$Proxy15.createExisting(Unknown Source)
      	at org.apache.ambari.server.state.cluster.ClusterImpl.loadServices(ClusterImpl.java:461)
      	at org.apache.ambari.server.state.cluster.ClusterImpl.<init>(ClusterImpl.java:352)
      	at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
      	at com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:260)
      	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
      	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
      	at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      	at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
      	at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)
      	at com.sun.proxy.$Proxy11.create(Unknown Source)
      	at org.apache.ambari.server.state.cluster.ClustersImpl.loadClustersAndHosts(ClustersImpl.java:194)
      	at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)
      	at org.apache.ambari.server.state.cluster.ClustersImpl$$FastClassByGuice$$7d58855f.invoke(<generated>)
      	at com.google.inject.internal.cglib.reflect.$FastMethod.invoke(FastMethod.java:53)
      	at com.google.inject.internal.SingleMethodInjector$1.invoke(SingleMethodInjector.java:56)
      	at com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:90)
      	at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
      	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
      	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
      	at com.google.inject.Scopes$1$1.get(Scopes.java:65)
      	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
      	at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
      	at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
      	at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
      	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
      	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
      	at com.google.inject.Scopes$1$1.get(Scopes.java:65)
      	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
      	at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
      	at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
      	at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
      	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:84)
      	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
      	at com.google.inject.Scopes$1$1.get(Scopes.java:65)
      	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
      	at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:38)
      	at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:62)
      	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:84)
      	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
      	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
      	at com.google.inject.Scopes$1$1.get(Scopes.java:65)
      	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
      	at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
      	at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
      	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
      	at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
      	... 7 more
      
      
      ERROR: Ambari server upgrade failed. Please look at /var/log/ambari-server/ambari-server.log, for more details.
      ERROR: Exiting with exit code 11. 
      REASON: Schema upgrade failed.
      

        Attachments

        1. AMBARI-20937.patch
          3 kB
          Vitaly Brodetskyi

          Issue Links

            Activity

              People

              • Assignee:
                vbrodetskyi Vitaly Brodetskyi
                Reporter:
                dereklwood Derek Wood
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: