Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-18184

Hive Metastore restart failed during EU with 'Internal credentials cache error' while running kinit

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: ambari-server
    • Labels:

      Description

      ambari-server --hash
      8250e90dc9ebcf1bd3dbac9b9eca8a6e21e073c9
      ambari-server-2.4.0.0-1127.x86_64

      Observed this issue in one EU run with below steps:

      1. Install HDP-2.4.0.0 cluster with Ambari 2.2.1.1 (secure, HA cluster)
      2. Upgrade Ambari to 2.4.0.0
      3. Perform EU to 2.4.2.0 and let it complete
      4. Start EU to 2.5.0.0

      Observed below error during Hive Metastore restart

      Traceback (most recent call last):
        File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 254, in <module>
          HiveMetastore().execute()
        File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 280, in execute
          method(env)
        File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 696, in restart
          self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
        File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 114, in pre_upgrade_restart
          self.upgrade_schema(env)
        File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 193, in upgrade_schema
          Execute(kinit_command,user=params.smokeuser)
        File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 155, in __init__
          self.env.run()
        File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 160, in run
          self.run_action(resource, action)
        File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 124, in run_action
          provider_action()
        File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 273, in action_run
          tries=self.resource.tries, try_sleep=self.resource.try_sleep)
        File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 71, in inner
          result = function(command, **kwargs)
        File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 93, in checked_call
          tries=tries, try_sleep=try_sleep)
        File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 141, in _call_wrapper
          result = _call(command, **kwargs_copy)
        File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 294, in _call
          raise Fail(err_msg)
      resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM; ' returned 1. kinit: Internal credentials cache error while storing credentials while getting initial credentials"
      

      A retry of the above failed task was successful and then EU proceeded to completion

        Attachments

        1. AMBARI-18184.patch
          3 kB
          Jonathan Hurley

          Issue Links

            Activity

              People

              • Assignee:
                jonathanhurley Jonathan Hurley
                Reporter:
                jonathanhurley Jonathan Hurley
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: