Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21512

Stack Advisor reported an error: KeyError: 'stack_name' while Issued INSTALLED as new state for NODEMANAGER

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.5.2
    • 2.5.2
    • stacks
    • None

    Description

      While In build #139, performing an [AUTHORIZED, HOST_ADD_DELETE_COMPONENTS] operation in RBAC for NODEMANAGER failed for AMBARI_ADMINISTRATOR role.

      Test Logs:

      2017-07-18 03:12:42,062|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|INFO: Requesting put on Request path : http://172.27.22.82:8080/api/v1/clusters/cl1/hosts/ctr-e134-1499953498516-15485-01-000003.hwx.site/host_components/NODEMANAGER
      2017-07-18 03:12:43,053|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|Jul 18, 2017 3:12:43 AM com.hwx.utils.logging.LogManager log
      2017-07-18 03:12:43,053|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|INFO: Response body : {
      2017-07-18 03:12:43,053|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"href" : "http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44",
      2017-07-18 03:12:43,053|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"Requests" : {
      2017-07-18 03:12:43,053|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"id" : 44,
      2017-07-18 03:12:43,053|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"status" : "Accepted"
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|}
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|}
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|Jul 18, 2017 3:12:43 AM com.hwx.utils.logging.LogManager log
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|INFO: Response body : {
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"href" : "http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44",
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"Requests" : {
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"id" : 44,
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|"status" : "Accepted"
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|}
      2017-07-18 03:12:43,054|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|}
      2017-07-18 03:12:43,055|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|Jul 18, 2017 3:12:43 AM com.hwx.utils.logging.LogManager log
      2017-07-18 03:12:43,055|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|INFO: Service URL : http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44
      2017-07-18 03:13:13,099|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|Jul 18, 2017 3:13:13 AM com.hwx.utils.logging.LogManager log
      2017-07-18 03:13:13,099|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|INFO: Wait for 30 seconds. Total Wait Time : 30 seconds
      2017-07-18 03:13:43,390|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|Jul 18, 2017 3:13:43 AM com.hwx.utils.logging.LogManager log
      2017-07-18 03:13:43,390|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|INFO: Wait for 30 seconds. Total Wait Time : 60 seconds
      2017-07-18 03:13:43,521|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|Jul 18, 2017 3:13:43 AM com.hwx.utils.logging.LogManager log
      2017-07-18 03:13:43,521|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|SEVERE: Failed task while Issued INSTALLED as new state for NODEMANAGER
      2017-07-18 03:13:43,655|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|Jul 18, 2017 3:13:43 AM com.hwx.utils.logging.LogManager log
      2017-07-18 03:13:43,655|INFO|MainThread|machine.py:159 - run()||GUID=2d76c48d-01f4-4740-bf4c-a0c9170ca246|SEVERE: Error while processing host component NODEMANAGER state change request to INSTALLED
      

      API Logs:

      03:12:41 AM 07-18-2017 ServiceManager INFO - Action :adding component NODEMANAGER to the host ctr-e134-1499953498516-15485-01-000003.hwx.site in the cluster completed successfully
      03:12:41 AM 07-18-2017 ServiceManager FINE - UrlPath : http://172.27.22.82:8080/api/v1/clusters
      03:12:41 AM 07-18-2017 ServiceManager FINE - API call url:/clusters
      03:12:41 AM 07-18-2017 ServiceManager FINE - API call type: get
      03:12:41 AM 07-18-2017 ServiceManager FINE - API call response:{
        "href" : "http://172.27.22.82:8080/api/v1/clusters",
        "items" : [
          {
            "href" : "http://172.27.22.82:8080/api/v1/clusters/cl1",
            "Clusters" : {
              "cluster_name" : "cl1",
              "version" : "HDP-2.6"
            }
          }
        ]
      }
      03:12:41 AM 07-18-2017 ServiceManager FINE - UrlPath : http://172.27.22.82:8080/api/v1/clusters
      03:12:41 AM 07-18-2017 ServiceManager FINE - API call url:/clusters
      03:12:41 AM 07-18-2017 ServiceManager FINE - API call type: get
      03:12:41 AM 07-18-2017 ServiceManager FINE - API call response:{
        "href" : "http://172.27.22.82:8080/api/v1/clusters",
        "items" : [
          {
            "href" : "http://172.27.22.82:8080/api/v1/clusters/cl1",
            "Clusters" : {
              "cluster_name" : "cl1",
              "version" : "HDP-2.6"
            }
          }
        ]
      }
      03:12:41 AM 07-18-2017 ServiceManager FINE - UrlPath : http://172.27.22.82:8080/api/v1/clusters
      03:12:42 AM 07-18-2017 ServiceManager FINE - API call url:/clusters
      03:12:42 AM 07-18-2017 ServiceManager FINE - API call type: get
      03:12:42 AM 07-18-2017 ServiceManager FINE - API call response:{
        "href" : "http://172.27.22.82:8080/api/v1/clusters",
        "items" : [
          {
            "href" : "http://172.27.22.82:8080/api/v1/clusters/cl1",
            "Clusters" : {
              "cluster_name" : "cl1",
              "version" : "HDP-2.6"
            }
          }
        ]
      }
      03:12:42 AM 07-18-2017 ServiceManager FINE - UrlPath : http://172.27.22.82:8080/api/v1/clusters/cl1/credentials/kdc.admin.credential
      03:12:42 AM 07-18-2017 ServiceManager FINE - API call url:http://172.27.22.82:8080/api/v1/clusters/cl1/credentials/kdc.admin.credential
      03:12:42 AM 07-18-2017 ServiceManager FINE - API call type: get
      03:12:42 AM 07-18-2017 ServiceManager FINE - API call response:{
        "href" : "http://172.27.22.82:8080/api/v1/clusters/cl1/credentials/kdc.admin.credential",
        "Credential" : {
          "alias" : "kdc.admin.credential",
          "cluster_name" : "cl1",
          "type" : "temporary"
        }
      }
      03:12:42 AM 07-18-2017 ServiceManager FINE - Status: INSTALLED
      03:12:42 AM 07-18-2017 ServiceManager INFO - Request body  : {
        "RequestInfo": {
      	"context": "Issued INSTALLED as new state for NODEMANAGER"
        },
      "HostRoles": 
      	{
      		"state": "INSTALLED"
      	}
      }
      03:12:42 AM 07-18-2017 ServiceManager INFO - Requesting put on Request path : http://172.27.22.82:8080/api/v1/clusters/cl1/hosts/ctr-e134-1499953498516-15485-01-000003.hwx.site/host_components/NODEMANAGER
      03:12:42 AM 07-18-2017 ServiceManager FINE - Request body : {
        "RequestInfo": {
      	"context": "Issued INSTALLED as new state for NODEMANAGER"
        },
      "HostRoles": 
      	{
      		"state": "INSTALLED"
      	}
      }
      03:12:42 AM 07-18-2017 ServiceManager FINE - Using username : admin1
      03:12:43 AM 07-18-2017 ServiceManager INFO - Response body : {
        "href" : "http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44",
        "Requests" : {
          "id" : 44,
          "status" : "Accepted"
        }
      }
      03:12:43 AM 07-18-2017 ServiceManager FINE - API username:admin1
      03:12:43 AM 07-18-2017 ServiceManager INFO - Response body : {
        "href" : "http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44",
        "Requests" : {
          "id" : 44,
          "status" : "Accepted"
        }
      }
      03:12:43 AM 07-18-2017 ServiceManager INFO - Service URL : http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44
      03:12:43 AM 07-18-2017 ServiceManager FINE - UrlPath : http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44
      03:12:43 AM 07-18-2017 ServiceManager FINE - Using username : admin1
      03:12:43 AM 07-18-2017 ServiceManager FINE - API username:admin1
      03:13:13 AM 07-18-2017 ServiceManager INFO - Wait for 30 seconds. Total Wait Time : 30 seconds
      03:13:13 AM 07-18-2017 ServiceManager FINE - UrlPath : http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44
      03:13:13 AM 07-18-2017 ServiceManager FINE - Using username : admin1
      03:13:13 AM 07-18-2017 ServiceManager FINE - API username:admin1
      03:13:43 AM 07-18-2017 ServiceManager INFO - Wait for 30 seconds. Total Wait Time : 60 seconds
      03:13:43 AM 07-18-2017 ServiceManager FINE - UrlPath : http://172.27.22.82:8080/api/v1/clusters/cl1/requests/44
      03:13:43 AM 07-18-2017 ServiceManager FINE - Using username : admin1
      03:13:43 AM 07-18-2017 ServiceManager FINE - API username:admin1
      03:13:43 AM 07-18-2017 ServiceManager SEVERE - Failed task while Issued INSTALLED as new state for NODEMANAGER
      

      /var/run/ambari-server/stack-recommendations/1/stackadvisor.err contains:

      Traceback (most recent call last):
        File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 166, in <module>
          main(sys.argv)
        File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 116, in main
          result = stackAdvisor.recommendConfigurations(services, hosts)
        File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", line 776, in recommendConfigurations
          calculation(configurations, clusterSummary, services, hosts)
        File "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.6/services/stack_advisor.py", line 550, in recommendHIVEConfigurations
          super(HDP26StackAdvisor, self).recommendHIVEConfigurations(configurations, clusterData, services, hosts)
        File "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.5/services/stack_advisor.py", line 727, in recommendHIVEConfigurations
          super(HDP25StackAdvisor, self).recommendHIVEConfigurations(configurations, clusterData, services, hosts)
        File "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.3/services/stack_advisor.py", line 219, in recommendHIVEConfigurations
          super(HDP23StackAdvisor, self).recommendHIVEConfigurations(configurations, clusterData, services, hosts)
        File "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.2/services/stack_advisor.py", line 375, in recommendHIVEConfigurations
          self.recommendYARNConfigurations(configurations, clusterData, services, hosts)
        File "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.6/services/stack_advisor.py", line 181, in recommendYARNConfigurations
          super(HDP26StackAdvisor, self).recommendYARNConfigurations(configurations, clusterData, services, hosts)
        File "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.5/services/stack_advisor.py", line 779, in recommendYARNConfigurations
          stack_root = self.getStackRoot(services)
        File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", line 1159, in getStackRoot
          stack_name = cluster_env["stack_name"]
      KeyError: 'stack_name'
      

      Attachments

        1. AMBARI-21512.patch
          0.8 kB
          Sumit Mohanty

        Activity

          People

            sumitmohanty Sumit Mohanty
            sjanardhan Srikanth Janardhan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: