Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21798

Oozie server crashes post migration after regen kerberos keytabs and restart all services



    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.5.2
    • Fix Version/s: 2.5.2
    • Component/s: None
    • Labels:


      This is specifically for the IOP *4.2.0 *to HDP migration

      Start with an IOP 4.2.0 cluster with both Oozie and Knox installed, enable Kerberos then run thru the migration. Once EU is finalized, regen Kerberos keytabs via Ambari web UI and check the checkbox to make Ambari restart all services as part of the regen process.

      Notice Oozie crashes after a successful restart with following error in the oozie.log file

      2017-08-23 12:39:48,480 FATAL Services:514 - SERVER[umiak2.fyre.ibm.com] E0550: Could not normalize host name [${host}], Could not resolve host [${host}], ${host}: Name or service not known 
      org.apache.oozie.service.ServiceException: E0550: Could not normalize host name [${host}], Could not resolve host [${host}], ${host}: Name or service not known

      This is because Knox kerberos.json in IOP 4.2.0 has oozie proxy setting as
      "oozie.service.ProxyUserService.proxyuser.knox.hosts": "${hosts}",

      In HDP 2.6.2, the same property is now
      "oozie.service.ProxyUserService.proxyuser.${knox-env/knox_user}.hosts": "${clusterHostInfo/knox_gateway_hosts}", where the ${hosts} placeholder is changed and Knox params.py no longer has the logic to calculate it .

      A manual fix is to change Kerboers config for Knox - update "oozie.service.ProxyUserService.proxyuser.knox.hosts" to have value "${clusterHostInfo/knox_gateway_hosts}" ( done via UI, this will kick off regen keytabs and restart services)


        1. AMBARI-21798_trunk_01.patch
          13 kB
          Robert Levas
        2. AMBARI-21798_branch-2.6_01.patch
          12 kB
          Robert Levas
        3. AMBARI-21798_branch-2.5_01.patch
          12 kB
          Robert Levas

          Issue Links



              • Assignee:
                rlevas Robert Levas
                dili Di Li
              • Votes:
                0 Vote for this issue
                3 Start watching this issue


                • Created: