Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-5066

Ambari Upgrade Test, restart HBase, HBase Client Install failed

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0
    • Component/s: None
    • Labels:
      None

      Description

      Upgraded from Ambari-1.4.1 Stack 2.0
      When running QE upgrade script, which simply installs old version of Ambari, deploys HDP then upgrades Ambari server and runs smoke tests, it failed at restart HBase service.

      The script uses API call to stop services:

      curl -H "X-Requested-By: ambari" -X PUT -u admin:admin http:/192.168.64.101:8080/api/v1/clusters/cl1/services?ServiceInfo/state=STARTED  -d '{"ServiceInfo": {"state" : "INSTALLED"}}'
      

      Output:

      {
        "href" : "http://192.168.64.101:8080/api/v1/clusters/cl1/requests/10",
        "Requests" : {
          "id" : 10,
          "status" : "InProgress"
        }
      }
      

      HBase region server failed to stop

      stderr: 
      Traceback (most recent call last):
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py", line 66, in <module>
          HbaseRegionServer().execute()
        File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 95, in execute
          method(env)
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_regionserver.py", line 48, in stop
          import params
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py", line 56, in <module>
          local_dir = substitute_vars(config['configurations']['hbase-site']['hbase.local.dir'], config['configurations']['hbase-site'])
        File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/substitute_vars.py", line 36, in substitute_vars
          match = pattern.search(result)
      TypeError: expected string or buffer
       stdout:
      None
      

      /var/lib/ambari-agent/data/errors-71.txt is attached.

      The when trying to restart HBase, HBase client failed to install:

      stderr: 
      Traceback (most recent call last):
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_client.py", line 43, in <module>
          HbaseClient().execute()
        File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 95, in execute
          method(env)
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_client.py", line 30, in install
          self.configure(env)
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HBASE/package/scripts/hbase_client.py", line 33, in configure
          import params
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HBASE/package/scripts/params.py", line 56, in <module>
          local_dir = substitute_vars(config['configurations']['hbase-site']['hbase.local.dir'], config['configurations']['hbase-site'])
        File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/substitute_vars.py", line 36, in substitute_vars
          match = pattern.search(result)
      TypeError: expected string or buffer
       stdout:
      2014-03-12 00:03:40,773 - Group['hadoop'] {}
      2014-03-12 00:03:40,775 - Modifying group hadoop
      2014-03-12 00:03:40,804 - Group['users'] {}
      2014-03-12 00:03:40,805 - Modifying group users
      2014-03-12 00:03:40,825 - Group['users'] {}
      2014-03-12 00:03:40,826 - Modifying group users
      2014-03-12 00:03:40,849 - User['ambari-qa'] {'gid': 'hadoop', 'groups': [u'users']}
      2014-03-12 00:03:40,849 - Modifying user ambari-qa
      2014-03-12 00:03:40,861 - User['tez'] {'gid': 'hadoop', 'groups': [u'users']}
      2014-03-12 00:03:40,862 - Adding user User['tez']
      2014-03-12 00:03:40,892 - File['/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
      2014-03-12 00:03:40,899 - Changing permission for /tmp/changeUid.sh from 755 to 555
      2014-03-12 00:03:40,899 - Execute['/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 2>/dev/null'] {'not_if': 'test $(id -u ambari-qa) -gt 1000'}
      2014-03-12 00:03:40,907 - Skipping Execute['/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 2>/dev/null'] due to not_if
      2014-03-12 00:03:40,907 - User['hbase'] {'gid': 'hadoop', 'groups': [u'hadoop']}
      2014-03-12 00:03:40,908 - Modifying user hbase
      2014-03-12 00:03:40,915 - File['/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
      2014-03-12 00:03:40,916 - Execute['/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/hadoop/hbase 2>/dev/null'] {'not_if': 'test $(id -u hbase) -gt 1000'}
      2014-03-12 00:03:40,923 - Skipping Execute['/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/hadoop/hbase 2>/dev/null'] due to not_if
      2014-03-12 00:03:40,924 - Group['nagios'] {}
      2014-03-12 00:03:40,924 - Modifying group nagios
      2014-03-12 00:03:40,948 - User['nagios'] {'gid': 'nagios'}
      2014-03-12 00:03:40,948 - Modifying user nagios
      2014-03-12 00:03:40,958 - User['hcat'] {'gid': 'hadoop'}
      2014-03-12 00:03:40,958 - Modifying user hcat
      2014-03-12 00:03:40,965 - User['hcat'] {'gid': 'hadoop'}
      2014-03-12 00:03:40,965 - Modifying user hcat
      2014-03-12 00:03:40,971 - User['hive'] {'gid': 'hadoop'}
      2014-03-12 00:03:40,971 - Modifying user hive
      2014-03-12 00:03:40,978 - User['yarn'] {'gid': 'hadoop'}
      2014-03-12 00:03:40,978 - Modifying user yarn
      2014-03-12 00:03:40,986 - Group['nobody'] {}
      2014-03-12 00:03:40,986 - Modifying group nobody
      2014-03-12 00:03:41,004 - Group['nobody'] {}
      2014-03-12 00:03:41,005 - Modifying group nobody
      2014-03-12 00:03:41,025 - User['nobody'] {'gid': 'hadoop', 'groups': [u'nobody']}
      2014-03-12 00:03:41,025 - Modifying user nobody
      2014-03-12 00:03:41,047 - User['nobody'] {'gid': 'hadoop', 'groups': [u'nobody']}
      2014-03-12 00:03:41,048 - Modifying user nobody
      2014-03-12 00:03:41,056 - User['hdfs'] {'gid': 'hadoop', 'groups': [u'hadoop']}
      2014-03-12 00:03:41,056 - Modifying user hdfs
      2014-03-12 00:03:41,065 - User['mapred'] {'gid': 'hadoop', 'groups': [u'hadoop']}
      2014-03-12 00:03:41,066 - Modifying user mapred
      2014-03-12 00:03:41,074 - User['zookeeper'] {'gid': 'hadoop'}
      2014-03-12 00:03:41,075 - Modifying user zookeeper
      2014-03-12 00:03:41,084 - Package['unzip'] {}
      2014-03-12 00:03:41,126 - Installing package unzip ('/usr/bin/yum -d 0 -e 0 -y install unzip')
      2014-03-12 00:03:43,299 - Package['net-snmp'] {}
      2014-03-12 00:03:43,316 - Skipping installing existent package net-snmp
      2014-03-12 00:03:43,317 - Package['net-snmp-utils'] {}
      2014-03-12 00:03:43,333 - Skipping installing existent package net-snmp-utils
      2014-03-12 00:03:43,425 - Repository['HDP-2.0.6'] {'action': ['create'], 'mirror_list': None, 'base_url': 'http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.0.6.0', 'repo_file_name': 'HDP'}
      2014-03-12 00:03:43,431 - File['/etc/yum.repos.d/HDP.repo'] {'content': InlineTemplate(...)}
      2014-03-12 00:03:43,433 - Writing File['/etc/yum.repos.d/HDP.repo'] because contents don't match
      2014-03-12 00:03:43,433 - Package['hbase'] {}
      2014-03-12 00:03:43,450 - Skipping installing existent package hbase
      

      see /var/lib/ambari-agent/data/errors-85.txt

        Attachments

        1. AMBARI-5066.patch
          1 kB
          Andrew Onischuk

          Issue Links

            Activity

              People

              • Assignee:
                aonishuk Andrew Onischuk
                Reporter:
                aonishuk Andrew Onischuk
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: