Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-14847

Concurrent kinit Commands Cause Alerts To Randomly Trigger

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.2.2
    • Component/s: ambari-agent
    • Labels:
      None

      Description

      The alerts framework on each Ambari Agent runs alerts in a threadpool when the job triggers. This can cause the following error to randomly appear and the alert to go CRITICAL:

       Connection failed to http://nat-rare-21-dvitiiuk-2-5.novalocal:8088 (Execution of '/usr/bin/kinit -l 5m -c /var/lib/ambari-agent/tmp/web_alert_cc_f3f99363c3b7d1667f1287ce3a35aa52 -kt /etc/security/keytabs/spnego.service.keytab HTTP/nat-rare-21-dvitiiuk-2-5.novalocal@EXAMPLE.COM > /dev/null' returned 1.
      
      kinit: Internal credentials cache error while storing credentials while getting initial credentials)
      

      The alerts would randomly go CRITICAL at the end of their ticket expiration time only to become OK again shortly after.

      The cause is that the kinit command being executed to create new credentials cannot be run concurrently for the same user.

        Attachments

        1. AMBARI-14847.patch
          14 kB
          Jonathan Hurley

          Issue Links

            Activity

              People

              • Assignee:
                jonathanhurley Jonathan Hurley
                Reporter:
                jonathanhurley Jonathan Hurley
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: