Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-14342

Error during Node Manager start after enabling security post Express Upgrade from 2.1 to 2.3.4

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.2.0
    • None
    • None

    Description

      Cluster: <http://172.22.111.210:8080/>
      Alive for next 24 hours (increase lifetime if required)

      *Steps*
      Setup cluster at HDP 2.1.15 using Ambari 2.2.0 (unsecure cluster)
      Modify relevant tables in DB
      Start Express Upgrade to 2.3.4 and let it finish
      Enable Kerberos on the cluster

      Result:
      Observed that while starting services, all Node Manager failed to start and
      gave below error:

      Traceback (most recent call last):
      File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py", line 153, in <module>
      Nodemanager().execute()
      File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
      method(env)
      File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py", line 50, in start
      self.configure(env) # FOR SECURITY
      File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py", line 56, in configure
      yarn(name="nodemanager")
      File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
      return fn(*args, **kwargs)
      File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py", line 128, in yarn
      content="Marker file to track first start after enabling/disabling security. "
      File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in _init_
      self.env.run()
      File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run
      self.run_action(resource, action)
      File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 121, in run_action
      provider_action()
      File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 87, in action_create
      raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname))
      resource_management.core.exceptions.Fail: Applying File['/var/lib/hadoop-yarn/nm_security_enabled'] failed, parent directory /var/lib/hadoop-yarn doesn't exist

      stdout - attached

      Complete artifacts at the time error was seen are here:
      <http://linux-jenkins.qe.hortonworks.com/home/jenkins/qe-artifacts/os-s11-3
      -jretus-baikaltom20unsecr/ambarieu-e2e-baikaltom20-nosec-noranger-noha-1449839
      498/artifacts/screenshots/com.hw.ambari.ui.tests.monitoring.TestAmbariSmokeFun
      ctionality/testF_enableSecurity/_11_12_4_42_Exceptions_appeared_during_Enablin
      g_Security/>

      EU Jenkins job: <http://linux-jenkins.qe.hortonworks.com:8080/job/Run-HDP-
      Tests/85136/consoleFull>

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            aonishuk Andrew Onischuk Assign to me
            aonishuk Andrew Onischuk
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment