Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21798

Oozie server crashes post migration after regen kerberos keytabs and restart all services

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.5.2
    • Fix Version/s: 2.5.2
    • Component/s: None
    • Labels:
      None

      Description

      This is specifically for the IOP *4.2.0 *to HDP migration

      Start with an IOP 4.2.0 cluster with both Oozie and Knox installed, enable Kerberos then run thru the migration. Once EU is finalized, regen Kerberos keytabs via Ambari web UI and check the checkbox to make Ambari restart all services as part of the regen process.

      Notice Oozie crashes after a successful restart with following error in the oozie.log file

      2017-08-23 12:39:48,480 FATAL Services:514 - SERVER[umiak2.fyre.ibm.com] E0550: Could not normalize host name [${host}], Could not resolve host [${host}], ${host}: Name or service not known 
      org.apache.oozie.service.ServiceException: E0550: Could not normalize host name [${host}], Could not resolve host [${host}], ${host}: Name or service not known
      

      This is because Knox kerberos.json in IOP 4.2.0 has oozie proxy setting as
      "oozie.service.ProxyUserService.proxyuser.knox.hosts": "${hosts}",

      In HDP 2.6.2, the same property is now
      "oozie.service.ProxyUserService.proxyuser.${knox-env/knox_user}.hosts": "${clusterHostInfo/knox_gateway_hosts}", where the ${hosts} placeholder is changed and Knox params.py no longer has the logic to calculate it .

      A manual fix is to change Kerboers config for Knox - update "oozie.service.ProxyUserService.proxyuser.knox.hosts" to have value "${clusterHostInfo/knox_gateway_hosts}" ( done via UI, this will kick off regen keytabs and restart services)

        Attachments

        1. AMBARI-21798_trunk_01.patch
          13 kB
          Robert Levas
        2. AMBARI-21798_branch-2.6_01.patch
          12 kB
          Robert Levas
        3. AMBARI-21798_branch-2.5_01.patch
          12 kB
          Robert Levas

          Issue Links

            Activity

              People

              • Assignee:
                rlevas Robert Levas
                Reporter:
                dili Di Li
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: