Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-12581

Host check/clean-up logic should clean up new packages and folder names

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.2
    • Component/s: None
    • Labels:
      None

      Description

      Few HDP-2.3 installation ran into problem when install performed on VMs that are not clean. One of the reported problem is:

      Traceback (most recent call last):
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 38, in <module>
          AfterInstallHook().execute()
        File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
          method(env)
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 35, in hook
          link_configs(self.stroutfile)
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py", line 91, in link_configs
          _link_configs(k, json_version, v)
        File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py", line 156, in _link_configs
          conf_select.select("HDP", package, version)
        File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py", line 241, in select
          shell.checked_call(get_cmd("set-conf-dir", package, version), logoutput=False, quiet=False, sudo=True)
        File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
          result = function(command, **kwargs)
        File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
          tries=tries, try_sleep=try_sleep)
        File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
          result = _call(command, **kwargs_copy)
        File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
          raise Fail(err_msg)
      resource_management.core.exceptions.Fail: Execution of 'conf-select set-conf-dir --package ranger-kms --stack-version 2.3.0.0-2557 --conf-version 0' returned 1. ranger-kms not installed or incorrect package name
      

      AMBARI-12515 is looking into making conf-select call more robust but host-clean up script (along with host checkup) need to be modified to handle the new folders introduced under "/etc" as well as "/usr/hdp/current" or "/usr/hdp/version".

      Suggestion from Nate Cole regarding cleaning up conf dirs.

      cd /etc
      remove accumulo ambari-* atlas hadoop-* hive* hive-* mahout phoenix ranger spark zookeeper falcon hadoop hbase kafka knox oozie ranger-* (ranger-admin, ranger-kms, ranger-usersync) slider sqoop storm pig flume hive-hcatalog
      

      The host check script should be modified to include these folders in the report so that the cleanup script can clean it.

      1. AMBARI-12581.branch2.1.2.patch
        4 kB
        Dmitry Lysnichenko
      2. AMBARI-12581.2.patch
        4 kB
        Dmitry Lysnichenko
      3. AMBARI-12581.patch
        3 kB
        Dmitry Lysnichenko

        Issue Links

          Activity

          Hide
          dmitriusan Dmitry Lysnichenko added a comment -

          Committed to trunk for now
          Commit to 2.1.2 is pending

          Show
          dmitriusan Dmitry Lysnichenko added a comment - Committed to trunk for now Commit to 2.1.2 is pending
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Ambari-trunk-Commit #3209 (See https://builds.apache.org/job/Ambari-trunk-Commit/3209/)
          AMBARI-12581. Host check/clean-up logic should clean up new packages and folder names (dlysnichenko) (dlysnichenko: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=fb28fe7ac286e679112a7da19f72cabbf3caa8ee)

          • ambari-agent/src/main/python/ambari_agent/HostInfo.py
          • ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py
          • ambari-server/src/main/resources/custom_actions/scripts/check_host.py
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Ambari-trunk-Commit #3209 (See https://builds.apache.org/job/Ambari-trunk-Commit/3209/ ) AMBARI-12581 . Host check/clean-up logic should clean up new packages and folder names (dlysnichenko) (dlysnichenko: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=fb28fe7ac286e679112a7da19f72cabbf3caa8ee ) ambari-agent/src/main/python/ambari_agent/HostInfo.py ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py ambari-server/src/main/resources/custom_actions/scripts/check_host.py
          Hide
          dmitriusan Dmitry Lysnichenko added a comment -

          Committed to branch-2.1 as well

          Show
          dmitriusan Dmitry Lysnichenko added a comment - Committed to branch-2.1 as well
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Ambari-branch-2.1 #465 (See https://builds.apache.org/job/Ambari-branch-2.1/465/)
          AMBARI-12581. Host check/clean-up logic should clean up new packages and folder names (dlysnichenko) (dlysnichenko: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3e9e182d6acdd48402e80529c384357125a3e406)

          • ambari-server/src/main/resources/custom_actions/scripts/check_host.py
          • ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py
          • ambari-agent/src/main/python/ambari_agent/HostInfo.py
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Ambari-branch-2.1 #465 (See https://builds.apache.org/job/Ambari-branch-2.1/465/ ) AMBARI-12581 . Host check/clean-up logic should clean up new packages and folder names (dlysnichenko) (dlysnichenko: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3e9e182d6acdd48402e80529c384357125a3e406 ) ambari-server/src/main/resources/custom_actions/scripts/check_host.py ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py ambari-agent/src/main/python/ambari_agent/HostInfo.py

            People

            • Assignee:
              dmitriusan Dmitry Lysnichenko
              Reporter:
              dmitriusan Dmitry Lysnichenko
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development